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PREVIOUS STUDIES 
lhrough the first decade of the present 
ry there was relatively little questioning 
the traditional classical requirements for 
liege entrance. In 1911, the Committee of 
Nine on the Articulation of High Schools and 
Colleges (6) recommended the requirements 
for college entrance which have since been 
more or less standard. These requirements 
consisted of nine specified units, three in Eng- 
lish, two in a single foreign language, two in 
mathematics, one in social science including 
history, and one in natural science. Two addi- 
tional units were to consist of advanced work 
in academic subjects to complete the require- 
ment of a second major of three units. Four 
units were left as a margin to meet individual 
needs. 

In 1919, Colebank (5: p. 149) stated the 
conviction which was growing among more 

liberal school men that 


‘No convincing evidence has made cer- 
tain that a four year secondary school 
course, consisting largely of quite formal 
studies in foreign language and mathe- 
natics, taught by the methods that have 
long been customary in preparatory and 
ther secondary schools, actually lays a 
better foundation for successful college 
work and for subsequent success in life 
than courses of a different character. Many 

leges insist on a certain number of units 

Latin or other foreign language. They 

ild be expected to present evidence of 

mvincing nature that the enforcing of 

requirement enables them to secure 
dents with best preparation for college 
rk, such results being due to the educa- 
nal value of the subject.” 


\bout this time a new basis for the selec- 


{ students appeared in the form of psy- 
aptitude 


logical and other tests. The 


attention of investigators was directed toward 
a number of factors possibly predictive of 
college success. Those chiefly studied were 
intelligence as measured by some test or bat- 
tery of tests, high school scholastic average 
or rank in class, and the amount of credit in 
specified high school fields. 


As might be expected, there was not entire 
unanimity in the findings regarding any 
factor. However, there appeared to be less 
evidence in support of the predictive value 
inherent in the amount of high school credit 
in any field than in support of the predictive 
value of either intelligence or high school 
rank. After a study of the scholastic records 
of 450 freshmen in Syracuse University, May 
(14: pp. 436, 437) concluded that 


“By knowing the intelligence of a stu- 
dent and knowing the time he spends in 
study we can predict his honor points with 
but slightly greater error than we could if 
we knew also the units he offered and the 
average grade he made in them . . . Aside 
from statistical treatment there are other 
considerations that lead us to believe that 
the amount of preparatory school or high 
school work is a very poor index of college 
success.” 


Gowen and Gooch (10), in a study of 927 
graduates of the University of Maine, found 
that, compared on either the amount or the 
quality of the work done in each, high school 
subjects were about equal to each other in 
predictive or training value for the freshman 
year of college. Odell (16) made an exten- 
sive study of about 2000 freshmen in a num- 
ber of colleges and found some predictive 
value with respect to college grades in the 
number of credits in specified high school 
subject fields. However, in almost every case 
the general high school average, the I.Q., o 
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both taken together, were uperior to spe ific 


ubject patterns in predictive \ ilue 


In 1923 Gebhardt (9g) found no significant 
dence, in a study of students at the Colo 
lo State Teacher College, to indicate that 
ny one subject or group of subjects in high 
100 eater value than another as 
eparation for college work. In 1924, Mae 
hail (13) selected twenty-seven pairs of stu- 
nts of equal rating on psychological tests, 
f which one member had taken four years 
f high school Latin and the other none. The 
average freshman grades favored the Latin 


members of the pairs but the difference was 
not statistically significant. 

Bolenbaugh and Proctor (1) found little 
difference between the academic achievements 
of two groups of Stanford University men, 
one of which had three or more units of voca- 
tional subjects in high school and the other 
no more than one. Brammell (2) made a sim- 
ilar analysis of the records of 163 freshmen 
entering the University of Washington in 
1925 with five or more vocational units and 
of 353 with no more than one non-academic 
credit. The first year average favored the 
academic group slightly but the four year 
averages were identical. 

Sorenson (19), in a study of the success 
of Northwestern University freshmen, con- 
cluded that three or four years of high school 
Latin were definitely predictive of college 
success. French and mathematics appeared 
to differentiate students less efficiently. Kent 
and Schreurs (12) studied the relation of 
mental alertness scores, rank by quarters in 
high school class, amount of high school Eng- 
lish and mathematics, and general first year 
college grades to success in college English 
and mathematics courses for 228 men and 
296 women at Northwestern University. They 
found the amount of high school English to 
have a slight negative value for predicting 
success in college composition. The amount 
of high school mathematics had positive value 
for predicting success in trigonometry but not 
in algebra. 

Brammell (3), in 1932, reporting on the 
findings of the National Survey of Secondary 
Education in regard to the entrance require- 
ments of 517 colleges and universities, listed 
the average prescription at about ten units. 
In the period from 1924 to the time of the 
Survey, 209 requirements in the academic 
fields had been decreased or eliminated. On 
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the other hand, 165 had been 
Many provisions for regular admissior 
than by high school transcript wer 
but that remained the sole method j 
cent of western colleges and the chi 
in most sections. 

he study of the significance 
elements of the high school patter 
lege success has not been dis 
Douglass (8) summarized studies up 
and reported on his own study at th 
sity of Oregon (7) of 387 graduat 
class of 1930. He found that (8 
“Not only was there little or no relat 
tween pattern of high school credits 
lege success, but apparently there is little r: 
lation between the mastery of any partir 
subject and college success.” He concluded 
(8: p. 147) that “The author of this stud, 
feels certain that, a few decades hence, ow 
present practice of selecting college entrant: 
on the basis of minimum credits in certair 
fields of high school credits will seem 
curious and inexplicable anomaly.” Fiv 
years later, however, he joined with Smit! 
(18: p. 695) in reporting their study at the 
University of Minnesota to say “. if there 
is any subject in high school the pursuanc: 
of which tends more than any other study t 
better marks in college, that field is Latin.’ 

Whelden’s intensive study of five succe 
sive classes at Yale indicated that greater 
quantities of Latin taken in private schools 
or after entering Yale College led to better 
scholarship in subjects in which there w 
identity or great similarity of elements. But 
(20: p. 497) “There was no evidence at al! 
of any value of a more extended study oi! 
Latin as an intellectual discipline, serving t 
extend the scope of intellectual capacity 
whatever field it might be applied.” On t! 
other hand, Ross (17) found that among 5: 
students in three classes at Allegheny Colleg: 
those having more high school language d 
better college work. 

Nelson (15) discovered that among 200 
students at Iowa State Teachers’ College for- 
eign language was the only field in which th 
amount of credit earned in high school showec 
a significant relationship to college achiev 
ment. Clark (4), in a study of 1431 womer 
and 1182 men at Northwestern Universit) 
found Latin to be the only high school subject 
having any predictive value for college 
success. 
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, study of first year achtévement of 861 
bers of the entering class of/1934 at the 
University of Iowa, Hammer (11) 
i that college success is more dependent 
n the qualities of the student than upon 
eh school subject pattern. Accumula- 
credit in a specific subject field had 
lation to college success in that field; 
general success which may be associated 
foreign language and mathematics was 
e easily explained in other ways than by 
ning the inherent value of the subject as 
leterminer of college success. 


SUMMARY OF PREVIOUS STUDIES 


Investigations of the importance of the 
gh school pattern of subjects for academic 
joices and success may be classified as 
llows: 
1. Studies relating credit in specific high 
hool subjects to general college average for 
civen number of semesters of college work, 
success in certain college fields, or to 


th 

\fter due allowances have been made for 
ibility differences, few of the relationships 
discovered prove significant, although some 
special claims are made for Latin, French, 


and mathematics. Most of the investigators 
finding such relationships are cautious in 
asserting that they are causal. Even in pair- 
ing studies it is questionable if all important 
factors other than those under consideration 
are perfectly equated. 

2. Studies of the general college standing 

students offering classical and vocational 
patterns. 

Such studies reveal few differences and 
those small, but those which appear to exist 
favor the classical groups. However, the neg- 
tive evidence from a number of extensive and 
carefully conducted studies in which no such 
lifferences were found cannot be overlooked. 

3. Studies of the relation of high school 
and previous college patterns to success in 
ertain professional colleges or divisions. 

Very little appears to have been done in 

field. There is some evidence that cer- 
tain high school courses contribute to success 
in engineering but that the pattern is unim- 
portant for law. 

4. Studies of the general intellectual or 
academic quality of students who express 
their greatest interest in, or actually enter 
certain specialized courses. 


VATTER PATTERNS 


Che results of such studies are contradic- 
tory. Contradictions may be due to the un- 
reliability of reported interests, to differences 
in admission requirements for various profes- 
sional curricula in institutions where studies 
were made, or to differences in techniques 
employed in the studies. Present evidence 
does not appear to establish real differences 
among students preparing for professions for 
which university training is required. 

PURPOSE OF THIS STUDY 

The purpose of the present study was to 
discover the characteristics of the pattern and 
quality of educational experiences which 
might identify each of several groups of uni- 
versity students. Those selected for the study 
were those majoring in English, law, medicine, 
and engineering. The investigation naturally 
divided into two parts, indicated by the ques- 
tions: 1. What are the characteristics of stu- 
dents of each of these groups in relation to 
their relative success in the curriculum 
chosen? 2. How do the characteristics of stu- 
dents in one group differ from those of 
students in the others? 


COLLECTION OF DATA 


Data for the study were secured from sev- 
eral types of permanent student records at the 
State University of Iowa. Most of these were 
located in the registrar’s office. The groups 
chosen were selected as representative of cur- 
ricula which differ considerably and which 
enroll a number of students sufficient to pro- 
vide enough cases without covering too great 
a span of years. The greatest number of 
classes represented in any group was thirteen. 

In this study, students were classified as 
successful if they completed the work of their 
chosen curriculum and received the appropri- 
ate degree. Students were classified as unsuc- 
cessful if they dropped the course before 
beginning the third year, with grades averag- 
ing less than C, or 2.0. This definition was 
used because total grade points equivalent to 
a C average are required for graduation from 
most colleges of the University. Unsuccessful 
students were placed, for purposes of analysis, 
in two supplementary groups, engineering and 
liberal arts. 


COMPOSITION OF THE GROUPS 


The total number of students included in 
this investigation was 1623. Of these, 280 
completed the four year course in liberal arts 
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with a major in English; 251 completed the 
course in law; 317, the medical course, and 
323 received degrees in engineering. The re- 
mainder included 176 students who were un- 
successful in engineering and 267 who were 
unsuccessful in the liberal arts course. 

There were no women in either the success- 
ful or the unsuccessful engineering group. 
Women numbered only two per cent of the 
law and medical groups. Twenty-five per cent 
of the unsuccessful liberal arts group and 71 
per cent of the English group were women. 
A relatively larger percentage of women than 
of men persisted in college attendance beyond 
the second year, since only one fourth of the 
unsuccessful liberal arts group consisted of 
women, as compared to slightly more than one 
third of the entire group entering at the same 
time 

Ace AT HicH SCHOOL GRADUATION 


The mean age of successful men students at 
high school graduation was about 17.7 years 
in each group. In all groups it was found that 
students who ranked high in their high school 
classes were, on the average, younger than 
inferior students at high school graduation. 
The mean age of women at high school grad- 
uation was below that of men in the same 
group. There were no students of either sex 
in the entire study who were graduated from 
high school below the age of fifteen or above 
the age of twenty-nine. 


Size or HicH ScHoOoL ATTENDED 


Size of high school attended appeared to 
have little influence upon choice among the 
university curricula included in this study. 
The school size distributions for the failing 
groups differed so little from the others that 
one may also conclude, in general, that the 
size of high school attended has little effect 
upon college success or failure. Sub-groups 
composed of students from large schools 
(over 600 students) showed an advantage 
over sub-groups from smaller schools in qual- 
ifying examination percentile ranks and in 
first year grades in college. This was true for 
ill major groups except English, in which the 
ranked above the small- 
chool group (under roo students) but below 

» group from schools of medium size. How- 
ever, these differences tended to disappear 
g the course, and law was the only field 
showed an 


large-school group 


durit 


which the large-school group 
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advantage in final grades. It seems reaso: 

to conclude from this, that although schoo! 
in general do differ by size in the preparation 
they give their students for writing entranc 
examinations and doing first year colleg 
work, such advantages or handicaps do ; 
persist throughout the college course. 


RANK IN HIGH SCHOOL CLasses 

The law, medicine, and engineering group 
differed little in the percentage of member 
ranking in each third of their high sch 
classes. The ratio of students of these groups 
sectioned by rank in high school class was 
approximately 60:30:10, from the highest 
third to the lowest. The English group aver 
aged a higher rank in high school, with 
ratio of 78:20:2. The unsuccessful group: 
resembled each other in this respect, with 
ratios approximating 20:50:30. From these 
ratios, it is evident that there is a relationshi; 
between class rank in high school and chances 
of success in college. 

It may be deduced from the ratios giver 
that rank in graduating class does influence 
the decision of students in regard to colleg: 
entrance. Even in the unsuccessful groups of 
college students, fewer than one third of the 
number had ranked in the lowest thirds of 
their respective high school graduating classes 
It is also evident that class rank by thirds 
offers no positive assurance of either success 
or failure in any of the areas studied, since 
low-third students were found in each suc- 
cessful group and high-third students in both 
unsuccessful groups. The latter circumstance 
may be explained by the fact that the defini- 
tion of unsuccessful students makes no allow 
ance for students of satisfactory ability wh 
are unable to remain in college or to do work 
of adequate quality because of ill health or 
other factors which cannot be determined 

DELAY IN COLLEGE ENTRANC! 

The smallest percentage of students delay 
ing college entrance after high school gradu 
ation was in the law group, and the highes' 
in the engineering groups. There appeared t 
be no particular reason for the difference. It 
is of some interest to note that about on 
fourth of all students who enter the Univer 
sity were out of school a semester or longe! 
after high school graduation. However, litt 
evidence appeared in this study to ind 
that this delay affected college success in 
way. 








SUBJECT 


QUALIFYING EXAMINATIONS 


ean qualifying examination percen- 

for the law, medicine, and success- 
gineering groups were 63, 62, and ¢ 

vely. This is as near equality as could 

ted. Variances were also equal. The 

nk of the English group was 71, sig- 

y higher than those of the other 

That of the unsuccessful engineering 

was 42; and of the unsuccessful liberal 

up, 31. The differences in the means 

essful and unsuccessful groups indicate 

the qualifying examination has some 

in index of college success or failure. 

vever, the fact that every successful group 

led students ranking in the lowest decile 

the qualifying examination shows that it is 

om absolute as a predictive instrument. 


RELATIVE SCHOLASTIC ABILITY 


The differences in the ratios of students 
king in the highest thirds of their high 
hool classes, in mean qualifying examination 
ercentile ranks, and in mean one year and 
two year grade point averages, all point to the 
English group as scholastically superior to the 
ther successful groups included in this study. 
There is evidence that this superiority is not 
: result of the preponderance of women in the 
English group in the fact that the men of the 
English group averaged higher on each of 
these measures than the women. The data 
which support the conclusion as to the supe- 
riority of the English group point just as 
lefinitely to the scholastic equality of the 
iw, medicine, and engineering groups. 


CoLLEGE GRADES IN REQUIRED SUBJECTS 


With few exceptions, the grade means of 
ill sub-groups in required courses in the first 
ind second years of college varied from the 
grade means of these groups in all subjects 
taken at the same time in the direction of a 

0, or C average. This leads one to question 
whether apparent differences in the abilities 
{ student groups in required subjects, as con- 
trasted with their abilities in all subjects con- 
sidered together, may not be misleading. 
C grades may be more common in these 
ourses, where large classes and routine in- 
struction make reliable measures of individual 
accomplishment difficult to obtain. A _pre- 
ponderance of C grades would have the effect 
| undergrading the superior students and 
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overgrading the inferior, which would explain 
the conditions noted. Achievement of out- 
standing grades would represent unusual 
promise in the field and might be extremely 


valuable in the selection of students in appro- 
priate majors, especially English. 


(CORRELATIONS 


Qualifying examination percentile rank 
was correlated with final grade averages tor 
English majors more highly than with grades 
of engineers, .55 and .43, respectively. Cor 
relations with grade averages in law and 
medicine were smaller, .35 and .33. First year 
grades showed correlations of .86 and .80 with 
total English and Engineering grades, but a 
spurious element is involved in the inclusion 
of first year grades in the total from which 
final averages were calculated. Correlations of 
first year grades with final grade averages in 
law and medicine, involving no spurious ele- 
ment, were .49 and .47. Correlations of two 
year pre-professional grades with final grade 
averages were .50 for law and .53 for med- 
icine. 

Size of high school attended showed no 
more than chance correlation with final grade 
averages in any field. Rank in high school 
class, however, showed significant correlations 
with final grade averages in every successful 
group. Values ranged from .40 in English to 
.25 for law, with that for engineering .26, and 
for medicine, .31. The latter was almost as 
high as the corresponding correlation of qual- 
ifying rank with final grade averages. 


HicH ScHooL SuBJECT MATTER PATTERNS 


Mean amounts of high school English 
ranged from 8.01 semesters for the English 
group to 7.66 for the engineering groups. The 
greatest difference was less than half a semes- 
ter. In language credit the English group also 
ranked first, with a mean of 5.12 semesters. 
The law and medical groups earned means of 
4.89 and 4.82 semesters, respectively; the 
successful engineering group, 3.63; the unsuc- 
cessful liberal arts group, 2.96; and the un- 
successful engineering group 2.83 semesters. 
The maximum difference in this case was two 
and one fourth semesters. 

Mean credit in social studies ranged from 
6.25 semesters for the law group to 5.64 for 
the successful engineering group. Mean math- 
ematics credit covered a range of a semester, 
from 5.85 semesters for successful engineering 
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students to 4.82 for English students. The 
engineering group was high and the law group 
low in mean amounts of credit in science, with 
5-16 and 4.28 semesters, respectively. Credit 
in the non-academic field ranged from a mean 
of 5.14 semesters for the unsuccessful liberal 
arts group to 2.96 for the medical group, a 
difference of more than two semesters. The 
average total of high school credits ranged 
from 31.90 semesters for the law group to 
31.08 for the unsuccessful engineering group. 

When the unsuccessful liberal arts group is 
compared with the three successful groups 
which began their courses in the College of 
Liberal Arts, it is seen that the mean subject 
matter pattern of the former group differs 
notably from the others only in the smaller 
amount of credit in language and the greater 
amount of credit in non-academic subjects 
earned in high school by the unsuccessful 
group. These differences were about two 
semesters of credit each. None of the differ- 
ences in mean amounts of credit in any field 
among the three successful groups was as 
great as one semester. If the engineering 
groups are added to the comparison, the great- 
est difference in mean amounts of language 
credit is extended to two and one fourth 
semesters and that in mathematics to more 
than a semester. Credit in non-academic sub- 
jects is still within the two semester range. 

Among all the groups considered, high 
school credit varied greatly only in the lan- 
guage, the mathematics, and the non-academi 
fields. The English, law, and medical groups 
earned the highest mean amounts of credit in 
language. The unsuccessful groups were char- 
acterized by greater amounts of non-academic 
credit. The engineering groups averaged more 
mathematics credit, and the law group a small 
amount more of social studies credit. The 
engineering and medical groups took, on the 
average, more high school science than the 
other groups. 

While these differences indicate that pos- 
sibly some preparation was made in high 
school for the curricula selected in college, 
both individual and sub-group variations in 
each successful group equalled or exceeded 
the differences in the means of the major 
groups. In some cases, the pattern of prepa- 
ration for superior students within a group 
differed from that of the lower ranking stu- 
dents of the same group in the same manner 
as that of the entire group differed from those 
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of other groups. This was not universally 
true, however, for the high ranking student: 
of the medical group averaged less credit jp 
high school science than the low ranking 
medical students. While high ranking 
dents in English earned more credit in for 
eign language than those ranking low, both 
sub-groups averaged less than the Englis! 
group as a whole. 

From the data presented, it appears that 
there is no pattern of high school subject 
matter that prepares students for success jr 
one of the fields covered by this study mor 
definitely than for success in any other. The 
sole possible exception to this is in the cas 
of additional mathematics and science {fo 
engineering students. On the other hand, it 
appears that greater amounts of language and 
mathematics credit, and smaller amounts of 
science and non-academic credit distinguish 
to some extent, the better from the poorer 
students in any field. Again, science for the 
engineering group must be excepted. Amount 
of credit in high school English and social! 
studies appear of small significance, possibly 
because there was little variation in credit in 
these fields. 

Relatively large curricular offerings in !a: 
guage and mathematics, coupled with reduc: 
college entrance and high school graduatio: 
requirements in these areas, and small requir: 
ments, along with increased offerings, in noi 
academic subjects, account for the possibilit 
of greater variation in the amounts of creui! 
in these fields offered for college entranct 
Since greater variation does occur in thos 
fields in which it is possible, it seems reasor 
able that greater weight should be given t 
the small variations noted in other fields. 


On this basis, it appears that students ente: 
ing engineering have, on the whole, made som 
special preparation for this curriculum by tak 
ing additional mathematics and science 
high school. Students who are unsuccesst! 
in engineering have, on the whole, less preps 
ration in mathematics than those who ai 
successful. Law students, on the other hand 
have tended to take more language and socia 
studies than the average for all students 
medical students more language and scienc 
and English students, more English and |a! 
guage. Even in the successful groups, ther: 
some evidence that the better students show 
greater differences in this respect than th 











It should be kept in mind, however, 


poorer 
that these differences, while consistent, are 
orobably too small to have statistical signifi- 
-ance, and might be accounted for by chance. 


VARIATIONS IN HIGH SCHOOL SCHOLARSHIP 


\n additional matter of interest in connec- 
tion with the high school subject matter pat- 
tern is the number of students ranking higher 

r lower in each high school field than in all 
high school subjects taken together. The 
greatest number of high ranking students in 
the English major group was in the English 
field in high school; in law, social studies; in 
the successful engineering group, mathe- 
matics; and in both unsuccessful groups, non- 
academic. The greatest number of low high 

ol grades for the English major group was 
| mathematics, with the greatest net number 
in science. In law, the most low high school 
grades were in mathematics; in medicine, lan- 
ruage: in both engineering groups, English; 
ind in the unsuccessful liberal arts group, 
mathematics, with the greatest net number in 
inguage. These results may show a trend to- 
ward better or poorer work in certain high 
school fields for members of different univer- 
ty groups, but in every successful group each 
high school field was represented by both high 
ind low grades. 





GENERAL CONCLUSIONS From Tus STUDY 


The general conclusions drawn from this 
tudy may be stated as follows: 

1. The existence of small variations in the 
mmounts of credit earned in high school fields 
in which requirements are high, and greater 
variations in fields with more liberal require- 
, indicates the possible presence of some 
relationship between the curriculum pursued 
in the university and the high school subject 
matter pattern. 

While the specific relationships between 
imounts of credit in various high school fields 
ind the choice of college curriculum are small, 
they are consistent among the four curricula 
included in this study. Since the possibilities 
f variation in the high school pattern are 
restricted, any variation for a considerable 
group of students suggests the possibility of 
the operation of some selective factor. It 
must not be assumed, however, that any rela- 
tionships which appeared were absolute or 
essential, as differences of this size might well 
ccur by chance. 


ments 
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2. The relative number of students receiv- 
ing higher or lower grades in certain high 
school fields than their average in all, points 
to the possibility of special relationships be- 
tween certain high schools and college fields. 

The piling up of frequencies of higher 
grades in certain high school fields and of 
lower grades in others to show a trend in the 
distribution of special interest or ability for 
a college group appears worthy of notice and 
may justify further study. However, the 
presence of many high and low grades in 
other high school fields necessitates extreme 
caution in advancing any generalizations on 
the subject on the basis of this study. 

3. The general scholastic ability of the stu- 
dent, as measured by rank in thirds of high 
school graduating class, rank on college en 
trance examinations, and grades in prelimi- 
nary college courses, appears to be more im- 
portant for success in any of the curricula 
included in this study than specific charac- 
teristics of the high school subject matter 
pattern. 

Every group included in the study offered 
satisfactory evidence for this conclusion. It 
must not be implied, however, that this gen- 
eralization could be applied to predict success 
or failure in an individual case. 


SUBSIDIARY CONCLUSIONS 


In addition to the general conclusions listed 
above, several subsidiary conclusions appear 
tenable: 

t. Successful English students, as a group, 
are superior in general scholastic ability 
to groups of successful students in law, 
medicine, and engineering. 

There are no real differences in general 


‘ 
scholastic ability among groups of suc- 
cessful students in law, medicine, and 
engineering. 

3. Unsuccessful students, as groups, are 


inferior in all measures of scholastic 
ability used in this study to correspond- 
ing groups of successful students. 

4. The overlapping of measures of scho- 
lastic ability for individuals in success- 
ful and unsuccessful groups, and for 
individuals ranking high and low scho- 
lastically within successful groups, indi- 
cates that indices of achievement in any 
of the curricula included in this study 
are extrernely unreliable when applied to 
individuals. 
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he presence, in all successful groups, of 


considerable 


numbers of 


students with 


educational characteristics quite similar 
to those of failing groups suggests that 
some quality, which might be called per- 


sistence, not revealed 


by any of the edu- 


cational measures recorded in this study, 
may be an important factor in college 
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THE STUDY 


From this study and others in the area 
which have been reported, it appears reason- 
ably safe to point to a few generalizations 
concerning the relationship of high school and 
ollege subject matter patterns as probabil- 
ities, even though complete proof may be 
lacking 

In the first place, students who take more 


high 


school language and 


mathematics are 


better students, as a group, at least in the first 
years of college, than those who take less. 
his may be explained in four ways: 1. A 
difference in the subject matter which gives 
inherently better background for any scho- 
lastic task to the first group; 2. Differences 


in the learning requirements and in the aca- 
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in law, medicine, or engineering 
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tically the same chances of success 
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tudy, in the opinion of the writer, 
uccess, in each group, of students who, 
ery measured index, should have failed. 
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een permitted membership in their pro- 
ns, and what their ultimate success may 
However, it is evident that these students 
sessed some combination of personal qual- 
; which enabled them to survive academic 
which would have discouraged most 
nts. The writer believes, on this evi- 
e, that one should be very cautious in 
ng dogmatically that any given profes- 
goal is impossible of achievement for 
student, although it may be seen as 
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findings of research definitely are not 
as to make it inadvisable to encourage 
i high school student to follow a major in- 
terest in a non-academic field, even though 
nay later enter college. It does seem ad- 
, however, that he should be required 
exert strong and sustained mental effort in 
the work that he does pursue and that he 
should not be shielded from the results of his 
scholastic limitations. A realistic view of his 
mental abilities, and training in meeting de- 
feat and persisting in spite of it may well be 
| greater value to a student of mediocre 
bility than any particular pattern of subject 
itter he may study. 
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THE STUDY HABITS OF FAILING AND SUCCESSFUL 
STUDENTS IN THE FIRST TWO YEARS OF COLLEGE 


C. W. BRowNn 
University of California 


INTRODUCTION 


\n accurate determination of the contribu- 

in of correct habits of study to college suc- 
ess must await the development of quantita- 
ive methods for measuring the various study 
-kills utilized in college. In the absence of a 
better method, the questionnaire technique 
was used in the present investigation in an 
effort to determine roughly the amount of 
iriation in study methods to be found 
‘among failing and successful students. The 
results to be presented are merely suggestive 
but they assist in getting a more complete 
lescription of the factors affecting the stu- 
lent’s ability to do work at the college level. 


CONSTRUCTION AND ADMINISTRATION OF 
THE QUESTIONNAIRE 


\ questionnaire was constructed with items 
vering such topics as the following: 

reading, (2) reviewing, (3) concentrat- 
(4) critical thinking, (5) note taking, 
6) preparing for examinations, and (7) 
promptness in completing assignments. After 
two revisions the questionnaire contained 64 
nee 

rhe questionnaire was filled out by 211 
sophomore students on probationary status in 
the College of Letters and Science’ and by 
(20 students in an elementary psychology 
ourse in the same college. The two groups 
were comparable in the number of semesters 
spent in the University and in the number of 
‘redit hours of work attempted. Ample time 
was allowed for everyone to answer all items. 


METHOD OF EVALUATING THE RESPONSES TO 
THE QUESTIONNAIRE 

fo make certain that there was no overlap 

etween the two groups of students relative 

to their college scholarship, the grade point 

for each individual was obtained 


verage 


rom the registrar of the University. Arbi- 
trary limits of —4 grade points? and +-4 
University of California, Berkeley 


2Cr : : : 
G ¥ points are assigned for each unit of credit as fol- 
ws 4 : 


B 2,C 1, D0, E and F —1. 
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grade points were set up, and the question- 
naires of all students whose college average 
fell within these limits were discarded. This 
step, although reducing the number of avail- 
able cases, resulted in two groups with a de- 
cided difference in their average college 
achievement. 

In order to find out if differences existed 
in the methods of study of the two groups the 
percentage in each group giving the same 
answer to each item was computed. The dif- 
ferences between these percentages, and the 
reliability of these differences, were deter- 
mined and used as a basis for evaluating the 
study methods. 

Most of the percentages to be considered 
in the following discussion are for what may 
be considered the psychologically correct an- 
swer. By “psychologically correct answer”’ is 
merely meant the answer which was given by 
the majority of graduate students in psychol- 
ogy who were asked to fill out the question- 
naire. For many of the items there is no 
“psychologically correct answer”. Disagree- 
ment concerning which of the possible answers 
to an item is correct was expected, but such 
disagreement does not invalidate the findings 
to be reported. 

It is not practicable to include a summary 
of the responses to all of the items in the 
present report. Instead a comparison of the 
failing and successful students will be made 
in terms of the larger aspects of study meth- 
ods already mentioned. Some of the items of 
the questionnaire were related to more than 
one of these aspects, and therefore may be in- 
cluded in more than one of the topics in the 
following discussion. 


READING HABITS 


In the table are found the data necessary 
for the evaluation of the different phases of 
study habits. Columns three and four contain 
the average percentage of each group giving 
the correct answers to all of the items in a 
given phase. In columns five, six and seven 
the critical ratios of the differences for indi- 
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ing mu individual item. 
However, the consistency of the small differ- 
énces in so many items supports the conclu- 
sion that the two groups do not differ signifi- 
cantly in the proportion of correct answers 
made to questions concerning their reading 
habits. In general the percentage of failing 
tudents reporting use of psychologically cor- 
rect reading habits does not differ significantly 
from the percentage so reporting from among 
students. 
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HABITS OF CONCENTRATION 


Eight items of the questionnaire were con- 
cerned with different aspects of the student’s 
attempt to concentrate on the material of his 
courses. In seven of the eight items the per- 
centage giving the correct answer is larger for 
the failing groups, but in only one of these 
items is the difference between the two groups 
Statistically significant. The mean difference 
in percentage for the eight items considered 
together is only 6, which is not statistically 
reliable. The findings indicate that correct 
habits of concentration are reported by fail- 
ing and successful students in about the same 
proportions 


ATTEMPTS OF FAILING AND SUCCESSFUL STU- 
DENTS TO EVALUATE CRITICALLY AND 
ANALYZE THE MATERIAL OF 
THEIR COURSES 


he percentage in each group giving the 
correct responses to the eleven items dealing 
with critical evaluation of course material is 
given in the table. In six of the items the 
larger percentage was found for the success- 
ful students, indicating a tendency for them 
to use methods of critical thinking more fre- 
quently than the students who are failing in 
their college studies. However, as was found 
for other aspects of study methods, the differ- 
ences tended to be small. Three of the eleven 


EXPERIMENTAL 
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differences are statistically signifi 
these favoring the failing students. 1 
favoring the successful students. Ij 
items are considered together and 
percentages computed, the diffe: 
proportions reporting psycholog 
answers is only .2 per cent more { 
ing than for the successful students 
support the conclusion that the ty 
are not significantly different in t 
proportion reporting correct answei 
items on critical evaluation. 


Hasits oF Note TAKIN 


Seven items covered different 
note taking in regard to both textb 
lecture material. The table gives the resy 
on this phase of study. For the most part t 
differences were small and statistical); 
nificant. In one question, the prop 
failing students giving the correct 
significantly greater than the prop: 
successful students. Results from t! 
items support the contention that the: 
difference in the two groups in respect 
proportion of students reporting the 
correct habits of note taking. 


MertTHOps USED IN PREPARING |} 
EXAMINATIONS 


Results from the seven items dealing 
preparation for examinations are given i 
table. Five of the differences favor the fa 
ing student. This might be expected from t 
fact that repeated failures would tend to stin 
ulate the failing student to find specifi 
of improving his performance on exa 
tions. However, of the four statistically re! 
able differences, two favor the successful st 
dents. Apparently there is no wide discre 
ancy in the proportion of the two groups rr- 
porting the use of psychologically corr 
methods in preparing for examinations 


METHODS OF REVIEWING 


+h 


On five of the ten items dealing with meth 
ods of reviewing lecture and text materials the 
larger proportions of correct answers wert 
given by the failing students and on the re- 
maining five items by the successful student 
Neither group is shown to be using the co 
rect methods significantly more frequent) 
than the other in nine of the ten items. De- 
spite the fact that most of the standard | 

















| ratios are larger than 1.5, the results do 
, warrant concluding that either group is 
-yperior to the other in methods of review. 


PROMPTNESS IN PREPARING ASSIGNMENTS 


Six items were concerned with the question 
; whether the two groups differed in the 
inctuality with which they completed the 
: simments of their courses. A larger per- 
entage of successful students tend to fall 
behind in their assignments, and a larger per- 
tage of the failing students prepare assign- 
ments ahead of schedule, but, in general, 
neither group shows marked superiority in the 
jistribution of their study time. 


GENERAL SUMMARY OF THE FINDINGS ON 
rHE SEVEN PHASES OF StuDY METHODS 


The object of the present study was to 
determine if the difficulties encountered by 
uilin ¢ students could be attributed to their 
utilization of unsound study habits. From the 
foregoing comparisons it is safe to conclude 
that, in general, the failing student reports 
ising correct study habits as frequently as 
loes the successful student. In fact, there is 
light evidence that more failing students 
than successful students have developed 
sound study habits. No doubt many failing 
students could improve their academic 
achievement by improving their methods of 
study, but from the results of the present 
study it cannot be concluded that the diffi- 
ulties encountered by failing students are in 
iny large measure attributable to their utiliz- 

f unsound methods of study. 


| \BSOLUTE PERCENTAGE OF FAILING 
STUDENTS USING CORRECT 
Stupy HaBItTs 


In the foregoing discussion the statement 
made that probably many failing students 

id improve their scholarship by improving 
their study habits. This is based on the fact 
that the percentages reporting the use of cor- 
tudy methods tended to fall between 

nd 75%, and that sometimes from 25% 

; reported using methods which can be 
ered psychologically unsound. These 
be shown to occur for items on 


there is little disagreement as to the 
sically correct answer. For instance, 
only 43% of failing students had 

| the practice of glancing through a 

r of examining the main points in 
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a chapter, before beginning a careful reading 
of it. There is little question but that either 
of these procedures of getting an overview of 
the whole chapter before reading it, is more 
sound psychologically than to start at the be- 
ginning of the chapter and carefully read it 
to the end. In item 16, in which the student 
is asked if he takes issue with the points of 
view presented in the lectures, only 34% of 
the failing students answered affirmatively. 
Again, in item 14, 34% of the failing students 
reported that they usually made their first 
review of their lecture notes just a day or so 
before an examination, rather than a short 
time after each lecture or during weekly re- 
views. In item 47, 37% of the failing students 
reported that they do not ask themselves 
questions about the material of an assignment 
while they are reading it. 

The answers to the items just presented, 
together with similar answers to many other 
items, indicate that many of the students fail- 
ing in their courses are not practicing sound 
study habits. No doubt improvement in their 
methods of study would be reflected in better 
grades. However, the same indictment can be 
made against the study habits of successful 
students, for large percentages of these stu- 
dents also are utilizing faulty study methods. 


LIMITATIONS OF THE RESULTS FROM THE 
QUESTIONNAIRE METHOD 


In interpreting the foregoing results the 
limitations of the method should be kept in 
mind. There was no way of determining the 
accuracy of the individual answers, and there- 
fore the reports may not describe the actual 
practices and habits of the students. It can 
be contended that the failing students would 
tend to falsify their reports to a greater de- 
gree than the successful students because of 
the pressure put upon them by their proba- 
tionary status. The truth or falsity of this 
contention can never be established, and 
therefore it must be assumed that the suc- 
cessful and failing students were equally 
truthful in their answers to the items of the 
questionnaire. 

The questionnaire method used here showed 
the number of failing and successful students 
reporting a given study habit. In most in- 
stances it did not obtain an estimate of the 
frequency with which the habit was used by 
the students, nor did it in any way get an 
estimate of the efficiency of the habit when 
it was used. Such limitations of the method 
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1. 51 99 
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rHE NATURE OF SOME OF THE DIFFICULTIES OF STUDENTS 
FAILING THE FIRST TWO YEARS OF COLLEGE 


{ \\ BROWN 


University 
Numerous investigations have 

ility to succeed in college is 

y many factors, not the least 

which are high school prepara- 

ding ability, and scholastic aptitude. 

these studies have used representa- 

ups of freshmen or sophomore college 
‘udents. The present study attempts to iso- 
1e of the characteristics of the rela- 
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ships existing between the three variables 
ntioned and college success in a group 
heavily weighted with students who were 
failing to achieve an acceptable level 
holarship in their college work. 


of 


Selection of the Subjects. Failing students 
were selected from among those who were 
receiving special attention from the dean of 
their college because of their low scholarship. 
\!l of the subjects used completed their first 
four semesters of work at the University.’ 
fo these were added students who were com- 
parable to the failing ones in terms of the 
1umber of semesters of work completed and 
the number of hours of course work essayed. 


Measurement of the Variables. From the 
registrar’s office the students’ high school and 
college records were obtained and evaluated 
relative to the quality of the work performed. 
Letter grades were translated into numerical 
scores by assigning the following numbers to 
each grade: A:3, B:2, C:1, D:o, and E or 
F:—1.* Average grade point ratings were 
determined for each student for his four years 
{ high school and his two years of college 
work. In addition, the high school achieve- 
ment was evaluated separately for the follow- 
ing fields of subject matter: English and lan- 
guage, mathematics and natural science, social 
science, and biological science. 

In studying the reading capacity of failing 
students it was desirable not only to learn 
their average reading ability, but to learn if 
iny reading deficiencies existed for particular 
of subject matter. None of the avail- 
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able reading tests for college students gave 
separate scores for the particular subject mat- 
ters which were to be investigated, so it was 
necessary to construct new tests. The reading 
selections chosen for the different subject mat- 
ters were: (1) for English and language, a 
selection on the exposition of the drama; 
(2) for mathematics and natural science, a 
selection on mechanical force; (3) for social 
science, a selection on Egyptian economic 
history; and (4) for biological science, a 
selection on respiration in lower organisms. 
These selections were taken from standard 
college texts which were in use or had been 
used recently in elementary courses in the 
respective fields of subject matter. The texts 
of the tests varied from 3 to 4 mimeographed 
pages. At the end of the reading selection in 
each test there was a set of from 20 to 24 
multiple choice questions dealing with the 
material of the selection. The score for each 
subject matter test was the number of ques- 
tions answered correctly. The score for the 
total reading test was the sum of the scores 
from the individual tests. 

The time allowed for each subtest was 12 
minutes. A preliminary buffer test was used 
to acquaint the student with the method to 
be followed in the remaining four tests. The 
student was required to read over a selection 
once before he began answering the questions; 
thereafter he could refer to the text again if 
he so desired. All of the details concerning 
the length of the selections, number of ques- 
tions, time of testing, scoring, etc., had been 
worked out and standardized previous to the 
present study. 

Scholastic aptitude was measured by means 
of the Ohio State University Psychological 
Examination, Form 19. It consisted of three 
subtests, an analogies test, an opposites test 
and a reading comprehension test. The sub- 
jects were given all the time they desired to 
complete the test, the average time required 
being about two and one-half hours. 

Reliability of the Measures. The reliability 
of the grades of the individuals participating 








in the present study was computed from the 
following formula:' 


R I 


in which «4? is the mean, for N individuals, of 
a". The term o ,? is the sampling variance 
of the mean of the errors of measurement for 
any given individual for the full number of 
semesters he has been in school. The term 
z,° represents the variance of the mean 
semester averages of N individuals. The reli- 
ability of the college grades was .88, and the 
reliability of the high school grades .75. These 
coefficients indicate reliabilities sufficiently 
high for the purposes of the present study. 

The reliabilities of the separate reading 
tests were computed by the odd-even tech- 
nique, using the Spearman—Brown prophesy 
formula. They varied from .79 to .go. The 
reliability of the total test was .92. The reli- 
abilities of the subtests of the Ohio State 
University Psychological Examination, as re- 
ported by the author of the test are between 
80 and .84. The reliability of the total test 
is .92. 

Division of Subjects into Four Scholarship 
Groups. In order to study the three factors 
of high school achievement, reading ability, 
and scholastic aptitude, at different levels of 
college scholarship, the subjects were divided 
into four groups on the basis of their scholar- 
ship for the four semesters of college work. 
Che grade point averages for the four groups, 
with the symbol assigned each group, follow: 
Group Fr consisted of the lowest failures, 
having a grade point average of .44 and merit- 
ing dismissal from the University; Group F2 
included students who did not merit dismissal 
but who were deficient in scholarship with a 
grade point average of .85; Group Cr was an 
average group with a scholarship of 1.18 grade 
points; and Group C2 was a superior group 
vith an average grade point standing of 1.98. 
the range of achievement represented in all 
an average of —.30 grade 
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ur groups is from 
nts to 
In the comparisons to follow, the perform- 
Group C1, which is the C-average 
roup, is used as a base, and the achievement 
the other three groups is expressed as a 
Crit t I I 


2.93 grade points. 
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deviation from the mean of the Cr groyp ; 
terms of the sigma units of the Cr dict. 
bution. 7 
Comparison of the Four Groups in Ter» 
of the Average College Scholarship. When :} 
college scholarship of Groups F1, F2, and 
is expressed as relative achievement in tery 
of the performance of Group C1, we find thy 
following results: the mean of Group F; 
5-69 sigma units below the mean of Groy 
C1, the mean of Group F2 is 2.54 sigma un‘: 
below this mean, and the mean of Group | 
is 6.15 sigma units above the Cr mean. It 
apparent that very large and significant qj; 
ferences in college scholarship exist betwee: 
the mean achievements of adjacent groups 
Comparison of the Four Scholarship Grout 
in Terms of the Average Performance in 
Three Experimental Variables. Fig. 1 repr 
sents graphically the relative achievement 
the different groups in the three variable: 
using the performance of the Cr group as th 
point of reference. Each of the failing group: 
is seen to have a performance much below 
that of the Cr group, but the amount of th 
deficiency is somewhat different for the thre: 
variables. In the C2 group the greatest sup 
riority is shown in high school achievement 
and the least superiority in reading ability 
and scholastic aptitude. 
Comparison of the Performance oj t/ 
Four Groups in the Different Types of Sul 
ject Matter in High School. Fig. 2 presents « 
comparison of the relative performance in th: 
four kinds of subject matter in high school 0’ 
the four scholarship groups. The range 
variation in relative performance in the dil 
ferent subjects is considerably greater in thy 
Fr group than in either the F2 or C2 groups 
The C2 group does not show uniform super 
ority to Group Cr in the four subject matter 
The two subject matters best differentiating 
the failing groups from the Cr group are Eng 
lish and language and social science. The su! 
jects of greatest differentiating capacity be 
tween Groups Cr and C2 are found in th 
mathematics and natural science category 
Comparison of the Performance of the I 
Scholarship Groups in the Reading 
Utilizing Different Types of Subject Mati 
Fig. 3 gives a graphical representation of 
relative performance of the four groups 
four subject matter reading tests. The rar 
of variation is greatest in the Fr group 
least in the C2 group. No one of the ot 
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groups agrees with the Cr group in their 
relative achievement in the different tests. 

The natural science test highly differen- 
tiates the C2 group from the Cr group, but 
between the F2 and Cr groups, and between 
the Fr and F2 groups, the natural science test 
bas the least differentiating capacity. The 
ooorest performance of the Cr group in both 
high school achievement and the reading tests 

curred in the field of mathematics and nat- 
vral science. This somewhat general tendency 
‘oward weakness in this field would increase 
the difference between Groups Cr and C2 and 
decrease the differences between C1 and the 
‘jiling groups, thus contributing to the con- 
itions just noted. Another possible reason 
may be suggested. In the natural science 
reading test there is present less of the speed 
factor and more of the power factor than in 
anv of the other time-limited tests. This con- 
dition might operate in this test to give the 
failing student a higher performance relative 
to the Cr group. 

Comparison of the Performance of the Four 
Groups in the Subtests of Scholastic Aptitude. 
The relative performance of the four groups 
in the aptitude subtests is given in Fig. 4. 
Variation in relative performance is large in 
both failing groups. The two vocabulary 
tests, opposites and analogies, highly differen- 
tiate Groups Fr and F2. These subtests show 
greater differences between these two groups 
than any of the other individual measures in 
the three variables under study. 

The Pattern of Achievement in the Four 
Types of Subject Matter in High School. In 
each of the variables under study the partic- 
ular measurement used involved several fact- 
ors, and it is important to note the variation 
in achievement among these factors in the 
four groups of subjects. The pattern of 
achievement in different types of subject mat- 
ter in high school is given in Fig. 5. In order 
to facilitate comparison a line has been drawn 
connecting the averages of each group; no 
function is intended. It is seen that the rela- 
tive achievement in the different subjects 
varies from one group to another. The pat- 
tern of Group C2 cannot be considered as 
typical, or to be expected from all levels of 
scholarship, because the higher averages are 
being depressed by the inclusion of large 
numbers of A grades, it being impossible for 
the individual to attain better than an A 
grade. No average in Group Cr is so affected; 
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thus, the curve of this group probably repre- 
sents the expected (normal) pattern of 
achievement. 


The question arises as to whether the varia- 
tions in the curve of Group Cr are due to 
chance, or reflect reliable differences in prepa- 
ration in the different subjects. Standard crit- 
ical ratios were computed for differences be- 
tween adjacent points on each of the curves. 
They form the basis for the following state- 
ments. In Group Cr preparation in mathe- 
matics and natural sciences is significantly 
inferior to preparation in all other subjects: 
therefore, the initial rise in the curve of C1 
reflects a real difference in quality of prepara- 
tion. The change in trend between social sci- 
ence and biological science is not significant. 
The characteristics in the pattern of high 
school achievement in the C-average group 
may now be stated as follows: the prepara- 
tion in mathematics and natural science is 
definitely inferior to the preparation in the 
other types of subject matter; the preparation 
in English and language, although superior to 
that in mathematics and natural science, is 
significantly inferior to the preparation in the 
two remaining subjects; and the difference in 
preparation in the social sciences and biolog- 
ical science subjects is not statistically sig- 
nificant. 

The curve for Group F2 closely follows the 
form of the curve for Cr. The first rise is 
statistically significant (C. R. 3.67). The 
difference between English and language and 
social science has a standard critical ratio of 
1.87 which is strong evidence that a real dif- 
ference in preparation in these two subjects 
might exist. The trend between social science 
and biological science varies slightly from 
that found in the curve for C1, but the change 
is not significant (C. R. .61). It would ap- 
pear that in general, the pattern of prepara- 
tion in high school subjects of Group F2 fol- 
lows rather closely the pattern of Group Cr. 

In the curve of Group Fr the general 
trends again approach those of the Cr group. 
The first rise is statistically significant (C. R. 
2.59). The second rise corresponds in general 
direction to the change in the other curves 
but is not significant (C. R. 1.27). The third 
portion of the curve, the rise between social 
science and biological science, although not 
significant (C. R. 1.49), is more pronounced 
than found for Group F2. With certain minor 
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variations the pattern of achievement in 
Group F1 corresponds to that of Group Cr. 
The present data are not sufficient for sta- 
tistically determining the degree to which the 
variation in achievement in the different high 
school subjects conditions achievement in col- 
lege; but regardless of what role the pattern 
of achievement plays in college study, it may 
be safely concluded that this role would be 
about the same for both failing and successful 
students. The principal differences between 
the several curves in Fig. 5 is not a difference 
in trends but a difference in levels of accom- 
plishment. The failing groups differ from the 
successful groups in terms of lower levels of 
achievement in all fields of subject matter. 

The Pattern of the Reading Performance 
of the Four Groups. Each of the reading tests 
was made long enough so that only a few 
perfect scores were attained. Also, sufficient 
time was allowed that no individual failed to 
reach and work on the questions. Every 
individual, then, performed within the range 
of the test, and the relative patterns of per- 
formance were not affected by factors similar 
to those operating in the evaluation of the 
high school preparation. Inasmuch as the raw 
scores on the various reading tests are not 
directly comparable, the average performance 
of each group was expressed as a deviation 
from the mean performance of Group Cr, 
using the sigma of Cr as the unit of measure- 
ment. Fig. 6 presents graphically the pattern 
of achievement of the four groups. The pat- 
tern of Cr appears as a straight line at the 
value 0.00. Any divergence from a straight 
line in the curves of Group F1, F2, or C2, can 
be considered an indication of a variation in 
pattern from that of Group Cr. The lines 
connecting the scores of a given group are 
drawn to facilitate Comparison, no function 
being represented. 

It is apparent that the pattern of perform- 
ance of the Fr group does not follow closely 
the pattern of Group Cr. The highest relative 
performance made by Group F1 was in the 
natural science test. Using the score in this 
test as a reference point, equality in relative 
achievement in the different reading tests can 
be represented by a straight line drawn par- 
allel to the curve of Cr. Any divergence from 
this line will indicate a deficiency in the rela- 
tive reading achievement. It is obvious that 
Group Fr is particularly deficient in the read- 
ing of materials in English and language, and 
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in social science. The tests in these fields wer; 
the two longest; they utilized vocabularie: 
and dealt with concepts that were much sim. 
pler than was true for either the natura! sci. 
ence or the biological science ‘tests. From 
these facts the conclusion readily follows tha: 
the individuals of the F1 group are extremely 
slow readers. The F2 group shows a ver 
marked deficiency in reading social scienc 
material. No doubt this latter deficiency 
primarily due to the low rates of reading 
these individuals. 

The Pattern of Achievement in th: 
lastic Aptitude Tests. Since the raw 
were not directly comparable, in order to 
the achievement in the different tests on { 
same coordinates, the performances of ea 
group were expressed as deviations from th 
means of the Cr group. Fig. 7 presents the 
relative achievement of the four groups on the 
three parts of the aptitude test. As Group Fi 
made their highest relative performanc: 
the opposites test, the scores on this test 
selected as a point of reference. The pattern 
to be expected from successful college stu 
can be considered to be the pattern of Group 
Cr. It will be noted that the pattern of ( 
diverges very little from this pattern, but that 
there are differences between the patterns | 
the failing groups and the C-average group 

From the data at hand it is net possible t 
determine if these variations in the patter 
scholastic aptitudes, as measured by the O 
State University Psychological Test, contri! 
ute to differentiating failing and successful 
students in college. However, it is very pro! 
able that such differences of pattern as have 
been noted are too small to be of importanc 
in conditioning college behavior. The general 
low level of performance in all of the apti- 
tudes, rather than a difference in the patter! 
of achievement among the several aptitudes 
seems to be the factor contributing to colleg: 
failure. 


Rate of Increase in Performance With In- 
crease in College Scholarship. Some notion of 
the variations in the rate of increase in the 
performance in the three variables with in- 
creasing college scholarship can be gained 
from Fig. 1. In all three of the variables the 
most rapid change takes place between the 
levels of the F2 and Cr groups. This finding 
points to the existence of conditioning factors 
which vary in importance at different levels 

+} 


of college scholarship. It appears that the 
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wer factors, functioning in high school 

hievement, still play an important role at 
the higher levels of college scholarship, as the 
lope of the curve for this variable still has 

high value between Cr and C2. 

Variations in the Rate of Increase in the 
Different Part Measures of the Three Vart- 
ibhles With Increase in the Level of College 
\chievement. A re-examination of Fig. 2 will 
reveal an important difference in the rate of 
ncrease in the high school scholarship at dif- 
ferent levels of college achievement. For all 
types of subject matter the curves change 
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much more rapidly from the grade point level 
.85 to the level 1.18, than at any other levels. 
This change in rate is another indication of 
the depression of the average scores of Groups 
F1 and F2 in each of the four subject matters. 

The true nature of the rate of change be- 
tween Groups Cr and C2 for biological sci- 
ence, social science, and English and language 
cannot be determined because of the large 
number of perfect grades made in these sub- 
jects, especially by Group C2. This factor is 
not found in mathematics and natural science. 
The rate of change in achievement in this sub- 
ject matter probably closely approximates the 
rate of change to be expected in the higher 
levels of college achievement. 

The more rapid change in performance be- 
tween F2 and C1 may be noted for each of 
the four time-limited reading tests by re- 
examining Fig. 3. This graph furnishes fur- 
ther proof that the more rapid change between 
F2 and Cr, noted in preceding analyses, can- 
not be attributed to some subject matter 
factor. Whatever the cause of this difference, 
it seems to be operating in all of the types of 
subject matter analyzed in the present study. 

From an examination of Fig. 4 it will be 
apparent that the change in scholastic apti- 
tude is not constant with equal units of in- 
crease in college scholarship, and that the 
rate of increase varies with the different tests. 
Although the difference in college scholarship 
between Groups C1 and C2 is greater than 
between any other two adjacent groups, the 
increase in the associated scholastic aptitude 
is less than that found between other groups. 
A large number of perfect scores in Group 
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maximal level of scholastic ability for their 
age. Increments of increase in test perform- 
ance will get smaller the closer an individual 
approaches his maximal level. The approach 
to the limits of the student’s scholastic abil- 
ity, as measured by these tests, not the 
shortness of the tests, probably accounts for 
the reduction in the slope of the curve at the 
upper levels of achievement. 

From the foregoing findings it is obvious 
that equal increments of increase in college 
scholarship are not associated with equal in- 
crements in performance on most of the part 
measures of the three variables being studied. 
Increase in college scholarship from a failing 
level to an average level is associated with 
the greatest increase in performance in most 








of the measures utilized in the present 
investigation. 
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Concluding Statements. From the preced- 
ing analysis of the difficulties of students 
failing the first two years of college, the fol- 
lowing conclusions may be drawn: 

1. In the three variables of high school 
achievement, reading ability, and scholastic 
aptitude, the most inferior performance of 
failing students is in the variable of high 
school achievement. 

2. Failing students achieve their highest 
relative performance in the variable of read- 
ing ability. 

3. When high school achievement in the 
four fields of subject matter is examined, the 
failing students are found to give their lowest 
relative performance in English and langu- 
age, and social science; and their highest 
relative performance in mathematics and 
natural science, and biological science. 

4. In the reading tests utilizing different 
types of subject matter the failing groups 
achieved their lowest relative performance in 
reading English and language material, and 
their highest relative performance in reading 
material in natural science. 

5. In the scholastic aptitude tests the 
lowest relative performance of the two fail- 
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g groups was made in reading compre- 
hension. 

6. The range of variation in performance 
in the different part measures of each of the 
three principal variables tended to be greater 
in the failing groups than in either of the 
successful groups. 

7. The pattern of achievement of the fail- 
ing groups in the different high school sub- 
iect matters did not vary greatly from the 
pattern of the successful groups. 

8. In the reading tests and scholastic apti- 
tude tests there was some evidence that the 
pattern of achievement of the failing groups 
differed from the pattern of the successful 


groups. 
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9. The most outstanding difference in the 
performance of the failing and successful 
groups was not a difference in pattern of 
achievement, but rather a difference in level 
of achievement, the performance of the fail- 
ing groups being depressed in all of the 
measures studied. 

10. The rate of increase in performance 
associated with increase in college scholar- 
ship was found to be higher for all measures 
between the levels of .85 and 1.18 grade 
points than between any other levels investi- 
gated. This most rapid change falls between 
the level of scholastic failure and the level 
of scholastic success. 





A STUDY OF THE CAVD INTELLIGENCE SCALE AND ITs 
RESULTS ON THE DOCTOR’S LEVEL AT COLORADO 
STATE COLLEGE OF EDUCATION’ 
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1. PROBLEM 


The object of this study is to determine 
the homogeneity of the I. E. R. Intelligence 
Seale CAVD, Levels M, N, O, P, and Q, and 
its validity for the purpose of predicting 
achievement at the doctor’s level in the Colo- 
rado State College of Education and to make 
comparisons of the abilities of students on 
the master’s and doctor’s levels in order to 
make a more adequate selection of candi- 
dates for work on the doctor’s level. The 
CAVD Scale is made up of four series of 
tasks: Completions, arithmetical problems, 


vocabulary, and directions. The upper five 
levels M, N, O, P, and Q are for use on the 
upper college and graduate levels. 


2. SUBJECTS AND TESTS 


The students matriculating on the doctor’s 
level from June, 1932 to August, 1938, were 
given the I. E. R. Intelligence Scale CAVD, 
Levels M, N, O, P, and Q, Form 4. In order 
to complete matriculation in 1936 and 1937, 
students on the doctor’s level were required 
to take the American Council on Education 
Cooperative English Test and a battery of 
twelve education tests. English Usage and 
Spelling were the parts of the English Test 
which were used. The battery of education 
tests consists of tests in the following sub- 
jects: Techniques of curriculum making, ele- 
mentary education, history of education, 
mental tests, personnel and guidance, phil- 
osophy of education, psychology of learning, 
research, school administration, secondary 
education, educational tests, and statistical 
methods. The tests were made up by various 
faculty members of Colorado State College 
of Education and were revised by the Per- 

' Based on Field Study No. 2, Colorado State College of 
Education, Greeley, Colorado, 1940. This study is a continu- 
ation of the study, “The Relationship between Scores on the 
CAVD Intelligence Scale and Success in Graduate Work at 
Colorado State College of Education,”” which was published 


in the March, 1940, issue of the Journal of Experimental 
Education 


sonnel Department of the college. There ar 
two equivalent forms of each test. Al! the 
items of each test are multiple-choice of four 
choices. The number of items in the various 
tests range from forty for research to fifty- 
seven for personnel and guidance. Data 0 
each test were compiled from the group oj 
ninety-one graduate students who first took 
the tests in the summer of 1936. 


3. RESULTS 


Of the students on the doctor’s level wh 
took the CAVD Intelligence Scale, 60 were 
in the graduate school at least two quarters 
and took a minimum of eight quarter-hours 
each quarter. The number of cases through- 
out the study is relatively small but no more 
were available; more reliable results might 
have been obtained from a larger number 

The reliability of teachers’ marks on the 
basis of the graduate work done by this group 
of 60 students in one quarter is .433 + .105 
(SE). This is a relatively low reliability. | 
order to compute the reliability of teachers 
marks and other correlations in which teach- 
ers’ marks are involved, the point-hour rati 
was found. The point-hour ratio was com- 
puted by dividing the total number of points 
by the total number of hours. The points 
were found by multiplying the number o! 
hours of each course by the point values of 
the letter grades. The point value of “A” i 
5; of “B” is 4; of “C”, 3; of “D”, 2; and 
of “F”, 1. 

The product-moment coefficient of correla- 
tion between scaled scores on the CAVD 
Intelligence Scale and the point-hour ratio 
based on the two quarters of the marks from 
which the reliability was computed is .52! 
+ .094. The coefficient of determination is 
.27; this coefficient gives the percentage 0! 
variance of teachers’ marks which is associ- 
ated with the CAVD Scale; the percentage 
of variance which is independent of the 
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CAVD Scale is .73. A half correction for 
attenuation was made in which the unreli- 
ability of teachers’ marks was corrected, but 
in which no correction was made for the un- 
reliability of the CAVD Test scores. This 
computation yielded a correlation of .670 
between the CAVD Test scores and the theo- 
retical true marks, which, rather than .521, 
is the measure of the true validity of the test 
when compared with a true criterion. The 
difference of the means of the point-hour 
ratio of the groups making above the 75th 
percentile (N == 49) and below the 25th per- 
centile (N = 15) on the CAVD is .704. 
Since the standard error of the difference is 

138, there is virtual certainty that there is 
a significant difference. 

In addition to a comparison of the results 

f scaled scores on the CAVD Intelligence 
Scale and teachers’ marks, a study was made 

f the relationship of each division of the 
CAVD Scale to teachers’ marks. The 
product-moment coefficients of correlation 

N 46) obtained between the total point- 
hour ratio of teachers’ marks and raw scores 
n the various divisions of the CAVD Scale 
are: (a) Completions, .491 + .112; (b) 
arithmetic, .136 + .145; (c) vocabulary, 
297 + .134; and (d) directions, .540 + 
ros. These correlations show substantial re- 
lationships between completions and direc- 
tions and teachers’ marks, slight relationship 
between vocabulary and teachers’ marks, 
and only negligible relationship between 
arithmetic and teachers’ marks. 

In order to complete matriculation in 1936 
and 1937, students on the doctor’s level were 
required to take the American Council on 
Education Cooperative English Test. The 
parts given were English Usage and Spelling. 
The product-moment coefficient (N = 54) 
between scaled scores on the CAVD Scale 
and scaled scores on the English Usage Test 
is .487 + .104; that between scaled scores 
on the CAVD and spelling, an r of .610 + 
086. These r’s show substantial relationships. 

In 1936 and 1937, students on the doctor’s 
level took Form A of the battery of twelve 
education tests. A product-moment coefficient 
of correlation (N = 48) of .691 + .076 was 
obtained between scaled scores on the CAVD 
Scale and the average of weighted scores on 
the education tests. This correlation indi- 
cates an improvement of 49 per cent over 
the assignment of scores by mere chance and 
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that the forecasting efficiency is 28 per cent. 
When this correlation was compared with 
the one for the students on the master’s level, 
it was found that 97 times in 100, the cor- 
relation for students on the doctor’s level will 
be the higher. 

A comparison of abilities of students on 
the master’s and doctor’s levels was made by 
computing the differences between the means 
and standard deviations and standard error 
of each difference for the following measures: 
(a) Scaled scores on the CAVD Scale, (b) 
first quarter point-hour ratio, (c) second 
quarter point-hour ratio, (d) point-hour 
ratio based on the first two quarters, (e) 
scaled scores on the English Usage Test, 
(f) scaled scores on the spelling test, and 
(g) average of the total weighted scores on 
the battery of education tests. 

The mean of the scaled scores on the 
CAVD Scale made by students on the doc- 
tor’s level is 7.338 + .884 higher than the 
mean made by students on the master’s level; 
the difference of the standard deviations is 
.604 + .625, with the SD of the masters the 
higher. Since the significance ratio (D/op) 
for the difference of the means is 8.30 there 
is positive assurance that, on the basis of the 
CAVD Scale, the doctors are always supe- 
rior to the masters. When the means of the 


three groups of point-hour ratios were sub- 
tracted, the means obtained from ratios made 
by the doctors were the higher in each case. 
The difference between the means of the 
point-hour ratios for the first quarter is .133 
+ .058. The difference between the means 
of the point-hour ratios for the second quar- 


ter is .265 + .072. And the difference be- 
tween the means of the point-hour ratios 
based on the first two quarters is .221 + 
.o61. The significance ratio of 2.30 indicates 
that, on the basis of teachers’ marks for the 
first quarter, doctors’ students are not always 
superior to the masters. The significance 
ratios of 3.68 and 3.63, for marks for the 
second quarter and for a composite ratio of 
the first two quarters, respectively, indicate 
virtual certainty that, on the basis of these 
marks, the doctors are always superior to the 
masters. None of the differences between the 
standard deviations of marks are significant. 

On the English Usage Test, the means and 
standard deviations of the masters are the 
higher. The difference between the ‘means is 
1.272 + 1.319; and the difference between 
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the standard deviations is 1.056 + .934. 
Neither of these differences is significant. 
The ratio (D/op) of the differences between 
the means and the standard deviations on 
the spelling test indicate even less assurance. 
When the means of the scores on the educa- 
tion tests were subtracted, that of the doctors 
was higher by 4.105 + .755. The significance 
ratio, 5.44, of this difference indicates with 
virtual certainty that, on the basis of the 
education tests, the doctors are superior to 
the masters. The standard deviation of scores 
on the education tests is higher by .643 + 
.§34, which is not a significant difference. 

In order to study the homogeneity of the 
upper five levels of the CAVD Scale, all pos- 
sible intercorrelations of its four parts were 
computed for 93 students on the doctor’s 
level. From the correlations, tre, = .378, 
T cay == .331, Toap == .251, and fcavn = 
.242, it is evident that the correlation between 
C and A is affected more appreciably by D 
than by V. The net correlation between C 
and V is affected very little when A is elim- 
inated, but D is evidently a factor in the 
correlation. This is shown by the following 
r’s: Tov = .534, Fov.a = -507, f cv.p = -414, 
and r cy.an == .409. The correlation between 
C and D is the highest of all, and indicates 
a substantial relationship between completions 
and directions. From the correlations, rep = 
553, Top.a == -491, Icpv = .442, and 
Tep.av == .386, evidently both A and V affect 
the correlation between C and D. The cor- 
relation of .193 indicates only a negligible 
relationship between arithmetic and vocabu- 
lary. The partials, ray —= —.OII, fay.p = 
.075, and ray.cp = —.033, denote the fur- 
ther reduction of the correlation between A 
and V when C, D, and both C and D are 
eliminated. The correlation of .327 between 
A and D denotes low relationship. From the 
partials, ranc == -153, Tap.v = .278, and 
Tap.cv == .156, C is a factor on which both A 
and D are dependent. The ryp = .392 is not 
appreciably lowered when A is partialled out 
(tvp.a == .355) but the elimination of C, 
which reduces the r to .137, indicates that 
vocabulary, directions, and completions are 
interdependent variables. 


4. CONCLUSIONS 
The I. E. R. Intelligence Scale CAVD, 


Levels M, N, O, P, and Q is a discriminating 
instrument for determining the range of abil- 
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ities of graduate students on the doctor's 
level. ; 

The correlation between the CAVD Scale 
and teachers’ marks is relatively low: hoy. 
ever, this correlation is consistent with th: 
low reliability of teachers’ marks on the doc. 
tor’s level. The CAVD Scale is sufficient}, 
discriminating to make it virtually certain 
that those students whose scores are aboy 
the 75th percentile on the CAVD will mak 
higher marks than those students whox 
scores on the CAVD lie below the 2sth per 
centile. 

Although they evidently measure differen 
abilities, substantial relationships were found 
between the CAVD Scale and the American 
Council on Education Cooperative English 
Usage Test and between the CAVD Scale 
and the American Council on Education Co- 
operative Spelling Test. There is a marked 
relationship between the CAVD Scale and the 
battery of education tests. 


On the basis of comparisons of scores on 
the CAVD Scale, teachers’ marks for the sec- 
ond quarter of graduate work, teachers’ marks 
for the first two quarters of graduate work, 
and scores on the education tests, students on 
the doctor’s level are always superior to stu- 
dents on the master’s level. The significance 
ratios of 8.30, 5.44, 3.68, and 3.63 for the 
differences between the means of the scores 
made by students on the doctor’s and mas- 
ter’s levels on the CAVD Scale, the battery 
of education tests, marks for the second 
quarter, and composite marks for the first 
and second quarters, indicate the varying 
degrees of assurance that the doctors are 
superior to the masters’ students. On the 
basis of the results obtained from these 
measures, a scale of values might be set up 
to be used in selecting candidates for the 
doctor’s degree. 

From the results of the intercorrelations of 
the four divisions of the CAVD Scale, the 
language tests (C, V, and D) are measures 
of different traits from those measured by the 
arithmetic test. Such correlation as appears 
between the language tests and arithmetic 
must be due to their common dependence 
upon reading comprehension. Evidently, A is 
measuring something very different from C 
and V and largely different from D. Since 
the divisions of the CAVD Scale do not over- 
lap very much, each part makes a unique 
contribution to the whole. 





ANALYSIS OF THE EFFECT UPON THE RELIABILITY 
COEFFICIENT OF CHANGES IN VARIABLES INVOLVED 
IN THE ESTIMATION OF TEST RELIABILITY 


TEOBALDO CASANOVA 
University of Puerto Rico 


I. THe VARIANCE OF THE HALVES IN THE 
SpLit-HALF METHOD 


For computational purposes the reliability 
of a test may be defined as the correlation 
between two equivalent forms. For this pur- 
pose two forms are said to be equivalent 
when their means and variances are equal— 
a situation that rarely, if ever, occurs in 
actual practice. As neither the differences 
nor the significance of the differences between 
the statistics of one form and those of the 
other form are usually reported along with 
the reliability of the test, we have been used 
to think about this reliability without much 
concern about the equivalence of the forms 
correlated. When two forms are available 
their correlation is obtained through the use 
of their actual statistics. Otherwise the reli- 
ability is estimated by the split-half method, 
which owes its popularity to the fact that 
only one form is necessary for the purpose. 

In this last method the correlation between 
the two halves of the available form is some- 
times computed from actually observed data, 
and from this the correlation of the available 
form with a hypothetical form is estimated 
through the Spearman—Brown formula. Here, 
even when the standard deviations of the two 
halves of the available form differ, the four 
standard deviations—two of the available 
half and two of the hypothetical half—are 
implicitly assumed to be equal when the 
Spearman—Brown formula is applied. This 
contradiction may be avoided through the 
use of one of the formulas given below. By 
another method in which the reliability is 
estimated in terms of the variances of the 
sums and differences of the available halves, 
the four standard deviations are also assumed 
to be equal. It is obvious, therefore, that the 
use of different methods in the presence of 
actual disparity in the variance of the halves 
leads to different results which make difficult 
the comparison and transmutation of reliabil- 
ities computed under varying circumstances. 


Let the subscripts 1 and 2 designate the 
scores on the two available halves of the test, 
and the subscripts J and // the scores on the 
corresponding halves of the hypothetical 
form. It is necessary to assume that o, = o 
and that o, = oy. But suppose that o, + a, 
and that o; + oy. It is also necessary to 
assume that the four intercorrelations are 
equal. Let each one of these be known as r, 
and let X, + X, = S,, and X; + Xun = 
S;. Then, 





hn r (o,* + 20,0, + o,’) 
X2)(xX mz ) ae =? 
(41 + ) I ul a, we 270,08, + os? 
or, 
r(o,+o,)* 


2 
Cs; 


(1) 


%s:s; = 


In practice r is the correlation between the 
two available halves, or the reliability of one- 
half of the test, and rs,s; is the estimated 
correlation of the available form with the 
hypothetical form, or the reliability of the 
whole test. 


(2) 


Ts, * mae 
i+fr 

which is the Spearman—Brown formula as 
used in the split-half method. Now S is the 
score on the sum of the two halves, or the 
score on the whole test. Let D be the score 
on the difference between the two halves. 
Then S=—7 + 2 and D=1— 2, and 
os" = o,” + 270,02 a @". (3) 


op* = o,° — 2ra,0, + o,°. 


Subtracting, 


os" == op? 


49,9, 


c= 


Substituting this value in (1), 


9 __ os* — ap" (9, + 42)? 
8,81 os? 40,0, 
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Since 


(o, + o,)? (a, — o,)* + 40,02, 


— |: — =| r+“ al) (5) 
? 4a wv. 


The form of this may be changed to 


afr =a | (e ap 
(oe, + a,)* 


substituting (4) into (2), 


(5a) 


Now, 


2 (os" — ap’) 
Tsisi 7: ps ¥ 


Cs” -Op + 4,0, 


and as from (3), 
ts’ + op* = 2 (a,” 


+ 4,02, 


— e"3 


2 (oe, —o,)? 
(6) 


I 
’ 


* (a, —9,)? 


— T aan 
| (o,—o,)*|- (6a) 
- T — 7 - — 


Os 


Formula (1), although never used, yields 
the most nearly correct estimate of the reli- 
ability of the test because it involves the least 
number of assumptions. It is easily seen that 
unless o, ==, its value is smaller than that 
given by (2), if its form is changed to 


r(o, +9,)? _ 


(a, —o,)* + 2e,0, ( I +7) 


(1a) 


Tsisi 


A common procedure is to find the correla- 
tion between the halves using the observed 
variances and then apply the Spearman- 
Brown formula. The value thus obtained is 
represented by formulas (6) and (6a), as 
may be inferred from their derivation. As the 
four variances are assumed to be equal in the 
derivation of the Spearman—Brown formula, 
there is a contradiction which may be avoided 
by using formulas (5) and (5a) which only 
involve the necessary assumptions that o, = 
o; and that o, = oy. (5) and (sa) are dif- 
ferent forms of (1). 
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Since 


(7, + o;)* > os* 

unless r == 1.00, the value given by (6a) 

larger than that given by (5a) unless co, —,. 

when the two values are equal. 
If o, = 2, each one of these formula 

becomes 


This formula, derived by John Flanagan: 
is now frequently used, especially when the 
tests are scored in the scoring machine, ly. 
cause the S’s and the D’s are read directly | 
the dial. Its derivation assumes that the fou 
variances are equal and its use, in case that 
a, + a, yields a smaller reliability than tha 
obtained from either (5) or (6). But it is no 
necessary to find op if the variance of one. 
half of the test is available. By using (; 
(7) may be turned into 
4qo,*F 


’sis1 = os? 


But this requires a knowledge of r. From 


I+? 





Then, 
I 2 


T3181 


Hence, 


Tsist 
r————_ 
2 — 188: 


Substituting this value in (8), 


20,” 


=2\,iI— 
Os 


Tsis1 
in which o,? is the variance of one-half of the 
test, and os’ is the variance of the whole test 
The formula is quite simple and the neces- 
sary data are easily available. The scores on 
one-half of the test are also read directly in 
the test scoring machine. It assumes that the 
four variances are equal, and its value | 
smaller than that given by either (5) or (6 


* Reported in Rulon, P. J. A simplified procedure ‘or 
termining 5 reliability of a test by split-halves. Harve 
Educ. Rev., 1 (1939), p. 101. 
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I]. THE CORRECTION FOR GUESSING 
| The Reliability of the Rights and the 
Reliability of tre Wrongs. The well known 








‘OEFFICIENT 





effect of changes in the several variables upon 
the coefficient of reliability of scores cor- 
rected for guessing, rx,x2.- 

















‘mula for correcting the scores in a mul- Let 
tiple choice test for guessing is, _— 
—— —a. 
. W TRiR2 
Ps — R —s mf (11) * «* ye . 
n—I Dividing both sides of (16) by reir, 
e ow ow” a 
or” — 2frw OR a-+ 1 
TxiXe __ S— TI (%— Zz }” 
° I7 
Trike 2 ow ow" (17) 
Cr’ — 2Trw OR : - 
(eas)  (8=-tP 





n which X is the score corrected for guessing, 
R is the number of right answers, W is the 
number of wrong answers, and m is the num- 
er of choices. Dichotomous scoring is as- 
imed throughout. Summing and dividing by 
VY, the number of cases, 





My 
My az Mp neem 9 (12) 
wt— I 
where the M’s are averages. If N =a, 
bec omes, 
W x 
Re~qaeees 
t (a—#) (13) 
Here X «is the true score after correction 


r guessing, Ro is the true score on the 
and Wa. is the true score on the 
ngs. Squaring, summing, and dividing by 
the infinite population in (13), 


hte 


wr 








2 
*1x,X2 == OR” TRyR2 — 27rw SRV lRiRe 
Ow oe “a y » 
V _— ————— ies. (14) 
n—I (n—1)* 


ms of the type ox? rx,x, represent the 
variance of true scores, and terms of the type 
rxx» are reliabilities. Squaring, summing, 
and dividing by the population in (rr), 

ow" 


—<S_—, (15) 


(n—1)* 


Ow 
a f 


4... 





on” — 2?rw Or 
n 


whence, 








Now, rrw is the correlation between the 
number of right answers and the number of 
wrong answers and it may be expected nor- 
mally to be negative. As dichotomous scoring 
has been assumed throughout, if each one of 
the subjects taking the test attempts to 
answer all the items, rrpw == —1.00. There- 
fore, assuming that rew is negative, it is 
obvious that 


Tx:X2 >; , OF < FRyR2; 


as 
> ,o, oF <, f. 


In such a situation it may be concluded that 
if the reliability of the rights and that of the 
wrongs are equal, the correction for guessing 
is unnecessary. If the reliability of the rights 
is greater than that of the wrongs, the reli- 
ability of the corrected scores is smaller than 
that of the rights, and the contrary is true 
in the opposite case. 

2. The Variance of the Rights and the 
Variance of the Wrongs. The reliability of 
the wrongs is not usually available, but the 
variance of the wrongs may be easily ob- 
tained. If the standard errors of measurement 





















: owV ow" Twiwe 
Or’ TRyR2— 2? rw ORV lRiR2— _ T W1W2 oo ee,  y 
n—1 ' * (n— 1)? 
rx \ A © 
: (16) 
> ow ow 
CR — 27Raw CR — ——+- > 
R—F (#n—T)* 


_ This formula will not be generally useful 
in estimating the reliability of scores corrected 


lor guessing because raw and rw,w, are not 
usually available, but it serves to show the 





of the rights and of the wrongs are assumed 
to be equal, the reliability of the wrongs can 
be eliminated from equation (17). Doing so, 
and letting 











222 
ow 
OR a P, 
I —VTaar 
2 ™ 12 : 
a Po om 18 
| Dees (18) 
Then, 
I —Vf re 
I —Twiwe + —_— 
p 
or, 
,—_ TriRe 
‘ww, = 1 —— >” 
p 
and, 
I 
y aoe 
TwiW2 I p- 
@” == ee ee oJ — (19) 
TRiRe p Trike 


Substituting this in (17) and dividing both 
members of the fraction on the right of (17) 
by or’, 
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depends respectively on whether the variance 
of the wrongs is greater, equal to or smaller 
than the variance of the rights. Thus the role 
of the variance of the rights is similar to thay 
of the reliability of the rights, and the role 
of the variance of the wrongs is similar ¢, 
that of the reliability of the wrongs, both of 
which would have been fair inferences +, 
draw from the assumption expressed in equa- 
tion (18). Since the presence of guessing jp 
a test is indicated by the presence of the 
wrongs, at least to the extent to which the 
variance of the wrongs is a measure of the 
quantity and intensity (whether mild or wild) 
of guessing, the above conclusions are jus. 
tified. One of these implies that if the vari. 
ance of the wrongs is exaggerated in compar. 
ison with that of the rights, the reliability of 
the uncorrected scores (that of the rights) 
suffers a shrinkage in comparison with that 














I — 2? rw 


2 lk 
I p—1\# I 
1+ + I 
TX:Xo —( TRiRe aaa 


2 
"Smet 
) 
Tr\Re 


(20) 





TRiRe 


2 


p 





p 
I — 2?rw 
Rw > + 


I (n— 1)? 


Here it is evident that if ra,n, == 7, ’x;x2 = 
whatever the value of the other vari- 
ables; and from (17) it follows immediately 
that rr»ro == fwyw2 = 1x,:x2. Hence, if the 
assumptions made are true, tests whose scores 
uncorrected for guessing have proved to be 
highly reliable need not be corrected for 
guessing. 

Equation (20) has been derived in order 
to study the dependence of the relative sizes 
of rx,;x» and rp,r, on the value of p*, which 
is the ratio of the variance of the wrongs to 
the variance of the rights. The last term in 
the numerator on the right of (20) can not 
be negative because it represents the ratio of 
two reliabilities. Subtracting from it the last 
term of the denominator on the same side, 


TriRe 





It is easy to see here that if rr,xn. < 7, the 
term of the numerator > , = , or < the 
term in the denominator as p > , ==, or < 7. 
Continuing the assumption that raw is nega- 
tive, if the other variables on the right of (20) 
remain constant, this leads directly to the 
conclusion that whether the size of rx,x, iS 
greater, equal to or smaller than that of reir, 





of the corrected scores, thus requiring the 
application of the guessing correction due to 
excessive guessing. 

The critical reader will not miss the impli- 
cation that if we make a similar comparison 
of rx,x. with rw,we by using the reciprocals 
of a and p in equations (17) and (20), the 
conclusions of this and the preceding subsec- 
tion would still stand, but with rw,we and 
Trin. interchanged. This fact is readily dis- 
closed by the symmetry of the right-hand side 
of (16) with respect to the two retiabilities 
and with respect to the two variances. How- 
ever, the comparison of rx,x, and fw,we is of 
no practical importance as the latter has 
never been used for any purpose. 





(21) 





Finally it is well to observe that if each 
one of the subjects attempts all the items o! 
the tests, while dichotomous scoring is used, 
Taw == —I, on = ow, and therefore, rx,x. = 
YniR2- This justifies the current practice of 
not correcting for guessing when all the sub- 
jects attempt all the items. 

















March, 1941] 


3. The Correlation of the Rights With the 
Wrongs. In order to study the effect of 
changes in frw, it is convenient to simplify 
the notation by putting A for the last term 
of the numerator of (20) and B for the last 
term of the denominator of the same equa- 
tion. Thus (20) becomes, 
axe __ 1 —afawVA TA (22) 
ayRe 1—2rrwVB+ B 
Assuming that raw is negative, it is impor- 
tant to know now whether 

4 B 
a A nna at ‘= , 
‘A+A 1r+2VB+8 


B and p + 1; or whether 





— 


I+ 2\ 


has a minimum value of 2 when A = 1, and 
the same applies to B or to any other posi- 
tive number. The value of this expression 
increases toward infinity both as A diminishes 
from 1 toward o, and as A increases from r 
to». It is most convenient to consider now 
nly the case when m = 2. Then B = p.? 
It was proved through equation (21) that if 
B>1,A > B> 1; and that if B < 7, 
{< B < 1. Therefore, unless B = 1, and 


in this case rx, x» == 7n,n2, nO matter what the 
value of Trw; 
A—1)? > (B—1)’, 


which means that A is more distant from 1 
than B, whatever the signs of the differences. 








It is therefore proved that if m == 2 and 
o> = #s 

2yA 2v5 . (23) 
r+2VA+A4 1+2VB+8B 
It is deduced from this that if rpw —= —1, 


the proportion of the coefficient of rrw in the 
numerator to the entire numerator of the 


r= op” 
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right-hand side of (20) is smaller than the 
proportion of the coefficient of raw in the 
denominator to the entire denominator of the 
same equation. As rrw rises from —z to 0 
in (20), if the other variables remain con- 
stant, the values of both members of the 
fraction decrease because rrw is negative; 
but from inequality (23) it follows that the 
value of the numerator decreases less rapidly, 
which results in an increase in the ratio on 
the left of (20). In a similar manner it can 
be shown that as rrw passes through o and 
continues its rise toward +1, the value of the 
ratio continues to increase until it reaches its 
maximum value when raw == +1. 

The foregoing discussion has not included 
the point of whether rx,x, > or < fpryr2- 
The ratio is determined by the values of a 
and p as indicated in the two preceding sec- 
tions. Thus, in case that rx,x. > frir2, the 
increase of rrzw from —zr to o exaggerates the 
discrepancy; when rx,x. < fr,r2, the same in- 
crease in rpw attenuates the discrepancy, and 
when 7rx,x. == 7r,r2, Changes in rpw do not 
affect the ratio. Then if a > 1 and p > 1, so 
that Tx1Xe > TryR2, a rise in Trw from the 
value of —z contributes to increase the dif- 
ference and to make more necessary the use 
of the correction for guessing. 

4. The Variance of the Number of Items 
Attempted. The correlation of the rights with 
the wrongs is not usually available, and for 
this reason equation (20) is mainly of theo- 
retical importance. By using equations (3) 
the value of rrw may be expressed in terms 
of the variances of sums and differences, 
which are more readily obtained, especially 
when the tests are scored in the test scoring 
machine. Changing r and 2 for W and R, 
and changing S for T we have from (4) that 

or? — ap" 


ay a es 
4orOw 


where T — R + W, and D = R — W;; that 
is, T is the total number of items attempted 
by one subject and D is the difference for 
each subject between his score on the rights 
and his score on the wrongs. Substituting this 
in (20), 





(4a) 








— y 4. a se ) < of fe 4.  eoet3 
TX1X2 2cn0w (n — 1) rriR2 (n—1)?* _TryRe (24) 
"Rake ae <r 2 
2onow (Nn — 1) (n— 1)? 

















Bearing in mind the changed notation, it is 
evident from (3) that as rew rises from —z 
to +s the value of or increases while the 
value of op decreases continuously for fixed 
values of ao, and ow. From the conclusions 
and assumptions of the preceding section it 
is inferred that an increase in oy produces an 
increase in the ratio on the left of (24). This 
is quite in agreement with current notions, 
according to which it is the fact that all the 
subjects do not attempt all the items that 
requires the use of the guessing correction. 
Since dichotomous scoring is assumed, oy” is 
the variance of the number of items attempted 
as well as the variance of the number of 
items missed. But while it is true that an in- 
crease in the value of or increases rx,x,. in 
relation to rp,n if the other variables remain 
constant, it is also true that this increase 
tends to make the two reliabilities equal if 
If p > 7, an increase in oy further in- 
creases the value of rx,x. over that of rp,ro, 
while if » == 7, changes in or do not affect the 
equality of rx,x, and rp,r2 if the assumptions 
made are correct. It is to be noted here again 
that if all the subjects attempt all the items, 
Tow -r, and from (3) it is evident that 
on ow. Then or = oO, and as it is well 
known, the guessing correction is unnecessary. 

5. The Number of Choices. If p = 1, 

Trine, Whatever the value of n. If 
1, then, returning to subsection 3, 
A < B < 1, and inequality (23) still holds 
with all its consequences. An increase in m 
would accelerate the rate at which a rising 
rew and a rising or increase the value of 
rx.x» over that of rp,x2. But since in these 
circumstances rx,x» < fr,Rr2, the effect of an 
increase in m is to diminish the difference 


between rx,x. and fpyre. 


p- I. 


2 

“Tc . p 
If p > 1, B, which is equal to yeaa 
may be > ,=,o<criHi B> }, 


A > B > 1, and inequality (23) holds again. 
Since rx,x» > fr re, an increase in m would 
tend to exaggerate the inequality. However, 
considering that m is stated in whole num- 
bers, and that this section is dealing with 
values of n or > 3, the cases where m — 2 


having already been treated in the preceding 


2 (or,*? — 
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sections, it is doubtful whether » can be j.. 
creased over the value of 3 to any consider. 
able extent while B remains greater thay 
or that is, when p > (n— 1). 

IfB <r while p > 1, we know tha: 


gam 8 . 
pan eS ae oe 8, 
TRiR2 


but A, which is equal to the quantity on th, 
left divided by (n — 1)*, may be > 
<u. If 


p* om J 
I -+——— J< (n—1 
( TRyR2 ): y 


A<1and B< A < zr. Here inequality 
(23) is reversed, and the effect of an in: reast 
in fgw and or is now to diminish the ratio 
’x,x2 tO fr yR2, While m remains large en 
so that (25) holds true. As now ry 


increase in m serves to exaggerate ‘tn dis- 
crepancy in A and B and thus to accelerate 
the effect of a rising rrw and a rising o« 
levelling down the difference between rx... 
and TRiRo- 

If A or > 1 while B < 7 and p 
equality (25) becomes an equation or it is r 
versed. In case it is reversed, B << 1 < A, an 
it is necessary to know which one of the tw 
variables is more distant from 1, in order t 
know whether inequality (23) or its reverss 
form hold. However, this case is of no great 
practical importance because, as we are dea 
ing with values of nm = or > 3, we are ver) 
unlikely to find a situation where inequalit 

25) does not hold. In conclusion, it may b 
said that, with the exception of some of the 
rare cases in which p has an exceedingly large 
value, which may upset inequality (25 
in the value of m has a tendency to equaliz 
Tx,x2 and rpyr2, which actually happens when 
no. 

6. Determinations in Terms of the Hal: 
The split-half method enables us to make 
formulas in terms of more easily availab| 
data for the estimation of the reliability o/ 
scores corrected for guessing, without going 
through the usual procedure. Returning t 
the notation of Section I, and preserving the 
meaning of R, W, and X in Section II, we 
have from formula (10), 


ow, ‘ ow,” 





27 RW OR 7 2 
ee (n—1) ° (n—1)? 





o R. — 27 


o Ws ows 
n—TI (s—1)* 





RsWs © Rs 

















11] 


bere the values of o, and gg in (10) have 
been substituted from (15). If the two vari- 


inces are equal in (3), 


2p," (1 + Yryr2/; 


by this and (9), 


r 


°(2- 


TFRs Rsl / 


4 


, similar manner, 


ows (2 — TWes, Wsy) 


(n 


(Ors 


nuing the Spearman—Brown assump- 
of Section I, 
4’ rw, TR Tw, 


TReOWs 


and substituting the values of op, and ow, as 


In (27) 


I _  ——— ee 
j ! 
> rsWs VI + frre VI + Pwiwe - (29) 


( 


> 


- 


Substituting into from 


28a), and (29), 


(26) 7), (28), 


( 


2 I 
— —TRsWs ORs 
o 


ORs — PRs Rs] ) 


. ove 


RELIABILITY COEFFICIENT 


This equation is almost equivalent to (16), 
although the value of the reliability coeffi- 
cient is slightly higher here if rrw is nega- 
tive. It could have been derived directly 
from (11) in terms of obtained scores, while 
(16) was derived from (12) in terms of true 
scores, on account of which the latter is more 
accurate and was preferred by the writer for 
the analysis made in the preceding section. 
From (3), 


ows) 


Owe OT (or. T 
(n—1) (n—1) 
Substituting from here and from (10) into 


(30), 


2TRs Ws Fits 


( Ors” 20w, / 


I) - 


7 Owes 


This is a simple formula which does not 
contain correlations or reliabilities, but it 
only gives a rough estimate of rxs, Xs) 
in case the assumptions made are not strictly 
fulfilled. For a closer estimate, (24) can be 
used. 

Ill. THe Errect oF CALLING ZERO ALL 

NEGATIVES SCORES 

Let X, designate a score of zero or a score 

above zero, X, a score below zero after cor- 





ows( 2 — Ws, Wey) 
(n—1)* 


ow. 


(n—1) 


Ren 





OwWs 


2 
Owes 





Or* 


— > . >." 
rs 2TRs nr 


n—1) 


Ows 


doen : 
' (n—1)? 


2 
Owa 





Ors” T Rs, — 217 RsWs ORs n | 


I ( 


>T Ws) We] 


"w— I }* 





2 
Ows 


(n—1)? 


Owe 
(n — 





Ons— 27 RsWsORs 
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rection for guessing, and X any score, whether If the measures are the sums of the score: 
on two equivalent halves of a test, all that jc 
necessary to express the relationship is to 
SX = TX, + TX, (32) change the notation. Thus, 


zero, above zero, or below zero. Then 


*: vf =r at (Mon? —2MsMs) ]. (35a) 


However, if the measures used are the differ. 

sX,? fax. + 2x} ences between the two halves, and if these 

(33) are assumed to have equal means and vari. 
ances, the means vanish, and 
This is the complete variance of the scores N 
on a test. Let ox,” represent the variance op.” = op? —— Onn". 
obtained when all the X, scores are called N 
zero. Now all the 6’s vanish, and since N, Substituting into (7) from (35a) and (26 


. 28x24 
nial N N? 


2Ms.Msy, | 


— 


2Mz,Ms a 


the number of cases is not reduced by calling in which r,is¢ is the reliability of the test 

zero the negative scores, after all negative scores have been called 
SX,? (3X,)? zero. 

TXe WV Ww?” From (7), 


Cos" ———JOp 





N N? N? 


' =X,’ (2X, )? 23X, 2X> 
Cxe ™ Cx -— 





If NV, is the number of & scores, and, 


3A," ( >X,)? Tsp" — Te 
Ny N,? Hence, 


“ = Ti Csp - 





V N, 
1, Lani esp" + - V ( Ms," — 2Mz,Msz, )] 

Mee. , Ne. (37) 
—Hle't Vs (Mas? meine 2Ms.Ms»)| 





r.1s is the reliability of the test computed 
—— see from all the scores, and 7,1, is the reliability 
a ae [ o? THy2 |: of the scores below zero after correction for 
: : iad guessing. The fraction on the right is a 
Substituting in (34), and letting NV, be the proper fraction and the quantity in brackets 
umber of @ scores, is positive if 7,» is positive, because M», 


Therefore, 





" ax ) 4 (2X0)? | 2N MN eM 


~ N2 tN? 
N, : 2N, 
—-"M, )-A2 MM] , 


Since N 
N, 
+ = M,,’ —2M.M,)] 





41] RELIABILITY 


which is the mean of the negative scores for 
he whole test is negative, while Ms,, the 
» for the scores of zero or above is posi- 
ve, Granting that 7, is positive and large, 
+ may be seen in (37) that rise < 1s, and 
an increase in the quantity in brackets 
rease the difference between 7,15. and 
. by diminishing 7,1se- Other changes that 
will jiminish fr,1rse are increases in N, the 
ber of negative scores, os, and Mgy. As 
is negative, it means that the further 
below zero the location of Msp, the smaller 
r... in relation to 7,15. In conclusion, it may 
be said that a considerable loss of reliability 
in the scores corrected for guessing may be 
expected after those scores that fall below 
zero have been eliminated, if the latter are 
many in number and spread far below zero. 


IV. THE VARIANCE OF THE ITEMS 


Kuder and Richardson’ have given the 
following formulas for estimating the reliabil- 
ty of a test in terms of the variance of the 
ems 


a, —n pq 
ee +e 


(38) 


(39) 


n which » is the number of items, o,? is the 
test variance, pg is the average item variance, 
p is the average p and gq is the average q, 
the proportion of persons missing one item; 
ind p+ q=vr. (39) is obtained from (38) 
by assuming that all the item variances are 
equal. But this never happens in test con- 
struction so that (39) is subject to an error, 
which may be of considerable size if there 
are wide differences among the item variances. 
[f the difference between the two values is D, 
subtracting the second from the first, 
2 
D = 


3(P 9 — 29). (40) 


(n—tI)o 


\s all the p’s are not equal, 7,, = —r. Also, 
'y ==, assuming dichotomous scoring. 
Then.? 


p q pq — See! 2) 
Kuder, G. F., and Richardson, M. W. 
t ef test reliability,’ Psychometrika, 


“The theory of the 
III (193 
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Since gp is positive it is evident that the value 
given by (38) is always larger than that 
given by (39), and 


ae n~ TD 


 it— 1) Co," 

By adding this to the right-hand side of (39), 
the value given by (38), which is the more 
nearly correct value, is obtained. 


ot —npg . 


2 


n* Tp 
(n—1) 0? 








(41) 


and since p+ q=—1, 


pI—=p(1—p) =p—P*. 
Substituting these in (41), 


CE) 


or 


n [ = PSP) 
B— J or” 


This simple formula, which can be obtained 
directly from (38), is more convenient than 
(38) for computing the value of 7,; because 
it does not involve the products. It shows 
that pq > o,”, as was to be expected in (41) 
from the fact that r,; << 1. The term pq can 
be changed in (41) without changing a,. 
This may be accomplished by adding a con- 
stant to each p. If the distribution of p’s is 
symmetrical with respect to p, it is evident 
that as the distribution of p’s is moved up 


and down the range, ?q varies, and reaches 


its maximum when p=.so. If the entire 
range of p’s is less than .50, it is possible to 
have the whole distribution of ’s either 


above or below .so. In either case p g would 
have a low value in comparison with the max- 


2This o p should not be confused with the standard error 
of the proportion in one category of a Bernoulli distribution. 





’ 


1 (41a) 
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(his means that, according an average difficulty near 7 or near I 

may be interpreted as being in conflict wis) 

Symonds’ well known findings® on the rely. 

conatens. he welts Geen ter las) os tion between item difficulty and reliat 
Abe ‘Seale eget: . but the point noted here is different 

mum when the mean of the distribution of = ¢5-muja (41) is correct, the present 

p's, which has been assumed to be symmet- sion jis unavoidable. It seems hardly nec, 

rical, is at .s0. That is, a distribution of ~’s_ sary to comment here that an increase 

with a mean of .50, which is the so-called either o, or m will result in an increase of 


mum value Of .25 
to formula (41), for a given o,, and if it is 
possible for the other variables to remain 


At Toe ? . ite “eS ; 2SS Symonds, P. M. “Choice of items for a test 
median difficulty of an item, produces a les: | le es FR 


reliable test than a distribution of p’s with pp. 481-493 





ITEM VALIDITIES OF THE OTIS SELF-ADMINISTERING TESTS 
OF MENTAL ABILITY FOR A COLLEGE POPULATION 


WILirAM R. Crooks 
Suffolk University 

and 

LEONARD W. FERGUSON 

University of Connecticut 


[his study was undertaken in order to pro- 
vide data concerning the difficulty and valid- 
ty of the items on the Otis test for a college 
nopulation comparable to those which Chap- 
anis' has furnished for adults. The popula- 
tion from which the results were secured was 
the student body at the University of Con- 
necticut. All tests were administered as labo- 
ratory exercises to the students (mostly fresh- 
men and sophomores) in elementary psychol- 
ogy. A description of the sample is provided 
n Table I. The data suggest that Form B is 


Chapanis. Examination of the data in this 
table reveals the following facts: 

1. The median difference for men between 
the per cent of adults and the per cent of 
students who got each item correct is .o8 
(Range .oo to .25). 

2. The corresponding median difference 
for women is .og (Range .or to .35). 

3. The median sex difference for Chapanis’ 
data in per cent of items correct is .o4 (Range 
.00 to .23). 


TABLE I 


DESCRIPTION OF THE SAMPLE 


N* 


100 
100 


Form A 
Men 
Women 


Form B 
Men 
Women 


137 
100 


Mean 
60.2 
57.7 


55. 0 
52.8 


Sex Diff. 


2.5.75 


Range S 
37-73 
40-72 


D 
.0 
7 


7.7 
10.1 


35-69 


22-70 2.2.81 


* There is no overlap among these groups; those students who took Form A did not take Form B. 


slightly more difficult from Form A. (The 
records for Form A were randomly selected 
from 1000 test records, but records for Form 
B represent all those available at the time of 
study.) It also appears that the test is either 
slightly more difficult for women or else that 
women are somewhat inferior to men in intel- 
ligence. Since so much evidence has been 
presented in refutation of the second alterna- 
tive, we are left with only the first, if the 
differences are interpreted as significant. 

In Table II are presented data showing the 
relative difficulties (per cent of correct re- 
sponses) and relative validities (biserial cor- 
relations) of all items in Form A of the Otis 
test. Presented for comparison with our own 
figures are those previously reported by 


A. “A note on the validity and difficulty of 
n Form A of the Otis Self-Administering Tests of 
Ability,” Journal of Experimental Education, V 
pp. 246-248. 


Chapanis, 


Ment a) 


> 
«- 


29 


4. The corresponding median for Crooks 
and Ferguson’s data is .03 (Range .oo to .29). 

5. The median difficulty of the items on 
the basis of Chapanis’ data is .86 for women 
and .79 for men. 

6. The corresponding median on the basis 
of Crooks and Ferguson’s data is .86 for 
women and .87 for men. 

7. The median item validity on the basis 
of Chapanis’ data is .61 for women and .62 
for men. 

8. The corresponding median on the basis 
of Crooks and Ferguson’s data is .48 for 
women and .46 for men. 

9. There are (according to the writers’ 
data) 18 items for college men and 23 items 
for college women which have non-significant 
validities. Nine of these are common to both 
men and women. 
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TABLE II 


VALIDITIES AND DIFFICULTIES OF THE ITEMS IN (HIGHER) FORM A OF THE OTI!s SeLr 
ADMINISTERING TEST FOR A COLLEGE AND AN ADULT POPULATION? 


Crooks-Ferguson Chapanis 
Men Women Men Womer 
Per Cent Per Cent Per Cent Per ( 
Correct Correct Correct r Correct 
100 10 . 9 96 .99 7 
96 a 92 ; 89 45 
90 .< an 86 .38 
GS a c 93 . 35 
93 06 : -_ 85 .46 
87 2 8! . of 86 . 30 
97 ! ¢ a 84 47 
89 on 92 ; 79 .42 
99 ; . 96 99 . 62 
} 97 . 51 
92 . 61 
93 88 
92 
80 
80 
93 
94 
90 
97 
87 
61 
87 
92 
80 
65 
94 
99 
91 
92 


UI -1 Ow 
IVDO DO PODS 


— tv 








ITEM VALIDITIES 


TABLE I]—Continued 


IDITIES AND DIFFICULTIES OF THE ITEMS IN (HIGHER) FORM A OF THE OTIS SELF- 
ADMINISTERING TEST FOR A COLLEGE AND AN ADULT POPULATION® 
Crooks-Ferguson Chapanis 
Men Women Men Women 
Per Cent Per Cent Per Cent ‘Per Cent 
r Correct Correct Correct r ‘orrect 
81 70 73 59 1 48 
79 68 : 61 
71 54 ‘ 45 
87 64 , 8 60 
60 46 a 31 
36 28 ‘ 20 
25 24 
43 if 32 
20 ) 18 
54 ) 47 
62 4 34 21 
76 4 41 28 25 
67 4 .6 37 . 66 34 5 29 
. 60 30 ) 15 . 78 16 . 12 


| 


4 


59 78 
45 74 
79 89 
62 67 


61 44 


oreo Or or em oe 
‘ >Or=1 Co 


66 41 8 27 
73 58 53 
42 28 Ys 33 
76 67 rf 64 
: 45 31 


»Dmuriw-IM-1-1 F 
“IS lO hee A 


RONHAS 
“oS 


[he data for the college population are ourown, Those for the adult population are from Chapanis. 


. There are (according to the writers’ which have non-significant validities and 22 
30 items for college men and 22 items items for college men and 27 items for col- 
llege women which have validities of | lege women which have validities of .50 or 
so or over. over (median difficulties are .34 and .47 for 
We may conclude from the above facts men and women respectively). 
that: (1) there is no difference in item diffi- We agree with Chapanis that with respect 
ity between adults and college students; to their validity certain of the items are mis- 
there are no particularly striking sex dif- ,Jaced, but because of inconsistencies between 
erences Eh eee difficulty either a= college Chapanis’, Otis’ and our data we are unable 
‘tudents or for adults; and (3) item valid-  ¢9 determine any one final correct order. It is 
ties tend to be somewhat lower for a college interesting to note that the items (59, 25, 20, 
croup than for an unselected group of adults. 29 and 6 of Form A) which Chapanis reports 
\ point of interest not brought out in the 45 the Jeast valid items for the population he 
ibove discussion is the apparent existence Of  ctudied are also among the non-valid items 
i tendency for item validity to vary inversely of our study (for either men or women or 
with item difficulty. Considering first the re- both). The items (1, 34, 72, 65, and 67 of 
jonses of men, the median difficulty for 30 form A) which he reports as most valid, 
“2 —- henge = binge MA however, are not among those which we have 
woe < ) é ->mS y -S “ . P ates 
validities is .95. The corresponding medians SE OREN ae Gear 
for women are in the first instance (based on 
22 items) .62, and in the second instance 
(based on 23 items) .92. Since, however. the 
non-valid items on any test are often those 
which are passed by more than 90% or less 
than 10% of a group, the actual relation may 


There are a number of items which appear 
in both forms A and B, in either identical 
wording or slightly modified wording but with 
the same essential content. For men the 
median difference in the item validities of 11 
of these items having significant correlations 
not be inverse. It may appear so here only ‘8 -09, and for women the median difference 


because there are so many easy items and so Of 12 items is .11. 
few hard ones. We shail not attempt to summarize the 
In Table III are given results secured on conclusions presented above, for that would 
Form B of the Otis Test. They appear to be involve mere repetition, but we should like 
quite similar to those already reported for to point out what we feel to be the most im- 
Form A. The median sex difference is .05; portant of them. These are that the validity 
the median item difficulty for men is .83 and and difficulty of the items in the Otis test are 
that for women is .77; there are 21 items for lower for a college group than for an adult 
college men and 14 items for college women _ population. 





JOURNAL OF EXPERIMENTAL EDUCATION 


TABLE III 


VALIDITIES AND DIFFICULTIES OF THE ITEMS IN (HIGHER) FORM B OF THE Oris 
SELF-ADMINISTERING TEST FOR A COLLEGE POPULATION 


Men Women Men Womer 
Per Cent Per Cent Per Cent Per 
Item No Correct Correct Item No. Correct C 
99 100 44 . 64 
85 5 74 45 ~ 51 
94 , 9] 46 ae 41 
92 79 47 . 06 83 
88 ; 77 48 .2 56 
73 3% 82 49 . 5S 70 
99 é : 50 . oe 84 
91 ) 51 : : 60 
94 2: ' 52 
100 53 
93 Dé 54 
92 f 55 


ANQNacouwh= 


Ome ee HID hoo 
© 


83 3 56 


vi 


~ 
Oo 
a-~ 


69 
70 
l 
9° 
3 
4 
5 


ie ee | 





THE PEABODY LIBRARY INFORMATION TEST: A STUDY OF 
ITS STATISTICAL VALIDITY AND RELIABILITY 


GrorcE H. DEER 
Louisiana State University 


Colleges and public schools alike have been 
-riticized for failure to teach their pupils an 
efficient and intelligent mastery in the use of 
the library. One probable factor in this tend- 
ency to neglect so important a function in the 
oupil’s growth and development has been the 
lack of an adequate measuring device for the 
skills and information involved. Until re- 
cently no satisfactory comprehensive test had 
been devised for use in the teaching of library 
nformation or library use. 


The Peabody Library Information Test, 
devised by Dr. Louis Shores, head of the 
library school, and Dr. Joseph E. Moore, pro- 
fessor of educational psychology, George Pea- 
body College for Teachers, contains 188 
multiple choice, completion, and matching 
items. In its revised form the test consists of 
eight sections, each of which is really a sepa- 
rate, independent test. These consist of (1) 
The Book, 24 items; (2) Arrangement of 
Books, 24; (3) The Card Catalogue, 24; 
(4) The Dictionary, 27; (5) The Encyclo- 
pedia, 24: (6) Periodicals and Indexes, 20; 
(7) Special Reference Books, 24; and (8) 
Bibliography, 23. 


Much as such a test is needed in this field, 
none can be accepted as desirable simply 
because it happens to be available. Careful 
users of a test will insist that its worth must 
be established by its validity and its reliabil- 
ity. This study is an attempt to examine the 
validity and the reliability of the Peabody 


Library Information Test when administered 
to 1322 students in nine colleges in situations 
involving actual classroom use. After all, the 
user of a test is not concerned with what 
might be the highest coefficient of reliability 
the author of a test has been able to estab- 
lish under ideal conditions, but he is inter- 
ested in the degree of reliability which may 
usually be expected for the test under every- 
day classroom conditions. 

The 1322 subjects for this test were ob- 
tained from the freshman classes of nine col- 
leges, as follows: Bethel College, 117; David 
Lipscomb College, 143; Fisk University, 111; 
Furman University, 241; Ouachita College, 
115; Peabody College, 121; Stetson Univer- 
sity, 198; Tennessee Polytechnic Institute, 
81; Western Kentucky Teachers College, 
135; and 59 juniors from Peabody College. 
The tests were administered in the regular 
English classes. 

Table 1 shows the number of students, 
measures of central tendency, and measures 
of variability for the Freshmen of each 
college. 


VALIDITY 


This study is not concerned with the pro- 
cedures used by the authors while construct- 
ing their instrument in order to insure its 
validity; it is concerned only with the sta- 
tistical measures which indicate validity or 
non-validity for the test when used in the 
classroom. 


TABLE 1 


A COMPARISON OF THE MEASURES OF CENTRAL TENDENCY, VARIABILITY, AND RANGE FOR THE 
FRESHMAN CLASSES OF NINE COLLEGES FOR SCORES MADE ON THE 
PEABODY LIBRARY INFORMATION TEST 


B 


143 
62.8 
50.3 
75. 0 
12.4 
64.4 
18.6 

1.5 

8-113 


Cc 


111 
61. 
51. 
74. 
|B 
63. 
17. 

z. 
18-98 


College 
Number Subjects -_- 
Median 


241 


6 1 


27272 
£39 


D 


58.4 
46.5 
72.8 
13.2 
60.5 
17.8 


23-123 


F 


121 
31.0 
21.4 
42.5 
10.5 
32.3 
15.8 

1.5 

5-62 


G 
198 


E 
115 


9.9 
51.9 
15.5 

1.5 

13-88 


1 
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TABLE 2 
DIFFERENCES BETWEEN THE MEANS FOR SCORES MADE BY FRESHMAN CLASSES OF NIN} 
ON THE PEABODY LIBRARY INFORMATION TEST. ALSO CRITICAL 
RATIOS FOR THESE DIFFERENCES 

y D a 
CR CR ; D 
2 12.0 3: é 52 
;9 2012.5 5.9 32 


0 1.0 10.6 8 30.5 


Me « 


19.2 


he first measure of validity applied to the two weeks’ period of instruction i 
test here is to determine whether it really of the library, presented in the regular | 
discriminates between the different groups English classes. This two weeks of ir 
to which it is administered. Table 2 presents increased the group’s mean from 47 
the differences between the means for each a difference of 49.6 with a critica 
of the nine colleges and the mean of every 
other college. Differences were found to be 
as high as 52.7 between the mean for school 
\ and that for school F. The column of crit- yerans, Sigmas AND CRITICAL RATI 
ical ratios is most significant in this table, FRESHMEN AND JUNIORS IN COLL! 
showing that 27 of the 36 differences meet ON THE PEABODY LIBRARY 

, ; INFORMATION TEST 


TABLE 3 


the general criterion of reliability, 3 sigma. 
[hese critical ratios run as high as to 24 for Freshmen 
differences of means for schools A and F. Number «a Sa 
This indicates that the test makes a high Mean --- 32.3 
degree of discrimination between groups in P 1 ania ha admaeaaanes — 
the different colleges. Difference between Means _- 

\ further test of discrimination was applied Standard Error of Difference 
by comparing scores made by the freshman _ Critical Ratio ----------- 
group in College F with those made by 59 
juniors in the same school who had no formal 20.2. This would seem to point to the 
library instruction in order to determine that the test was measuring, at least, what 
whether the test would discriminate between ever the instructor was teaching as lil 
freshmen and more advanced classes. One usage. These data are presented in Tab! 
would naturally expect that in two years of 
study the college student would acquire in- 
cidentally a considerable amount of informa- 
tion about the library and its use. A test (Comparison oF ScoRES MADE BY COLLEG! 
sensitive to different degrees of knowledge Juniors WiTH ScorES MADE BY THE SAM! 
should be able to recognize this additional INDIVIDUALS AFTER Two WEEKS OF 

Fj caf : wena LIBRARY INSTRUCTION 

amount of library information acquired be- 
tween the freshman and the junior year. Re- - Retest Atte 
sults of this comparison are presented in — ae Shade 
Table 3, which shows a difference of 15.6 in emia at Qebieais —, , 
the means for the two groups, with a critical oan Score ___ 47.9 
ratio of 6. This indicates a rather distinct gtandard Deviations... 16.6 
differentiation between classes at different Standard Error of the 
college levels. Mean 2.16 

As a further check on the ability of the Difference between Means —- 


bgp -eofter Correlation, initial and final 
test to make discriminations, these scores for a aR eet NR OG o 


the college juniors were compared with the Standard Error of Dy: - w:-- 
scores made by the same individuals after a Critical Ratio 


TABLE 4 


#t¢ 








PEABODY LIBRARY 


INFORMATION 


TEST 


TABLE 5 


1F FRESHMEN IN THE SCHOOLS 


LISTED HORIZONTALLY 


WHO EQUAL OR EXCEED THE 


MEDIAN SCORE OF THE FRESHMEN IN EACH SCHOOL LISTED VERTICALLY 


Cc 
9 
46 


59 
sv) 
98 

9 
43 
69 


21 


D E G 
e) : l 
41 5 
44 


68 
96 
40 
59 
18 


TABLE 6 


IRRELATIONS OF THE 


EIGHT PARTS OF THE PEABODY 


LIBRARY INFORMATION TEST ON 


SCORES MADE BY 117 FRESHMEN IN COLLEGE A* 


I] III 


9 9 
00 ~ 
) 


oe 


coefficient has a probable error smaller than .05 nor larger than 


mmonly used measure of the degree 
crimination a test possesses is the per- 
overlapping from one group to an- 
\ fifty percent overlapping for one 
over the median of another would indi- 
that the test was not measuring a differ- 
between the groups. 
ible 5 shows an overlapping ranging from 
to one hundred percent between the 
ferent colleges. Not a single student in one 
re ranked as high as the median of a 
n other college and not a student in this 
d school scored as low as the median in 
first. 
a test made up of several parts, as this 
is, it is important to know whether the 
separate parts of the test are really inde- 
ndent and measuring different things; oth- 
wise its great length cannot be justified. To 
etermine this, intercorrelations were com- 
| for all of the parts, using the students 
College A as subjects. This group was 
sen rather than another because of their 
h higher total scores on the test. The 
eficients in Table 6 denote relatively little 
relation between the different sections of the 
test. Each of the sections is measuring a 
lifferent function which cannot be measured 
by any one of the other parts. 


IV V 
. 33 24 39 


VII VIII 

26 31 
36 1] 

15 25 a 
27 .44 
34 


VI 


36 26 


06. 


RELIABILITY 

As a measure of the reliability of the Pea- 
body Test a comparison was made of scores 
obtained on alternate items. This was done 
by correlating the scores made on the even 
items with those made on the odd items and 
then applying the Spearman-—Brown correc- 
tion formula. Corrected coefficients were 
obtained as follows: Peabody Freshmen, .68; 
Peabody Juniors, .82; Peabody Juniors after 
two-weeks teaching, .96; T.P.I., .81; Oua- 
chita College, .92; Bethel College, .88. 

The probable error of estimate of a raw 
score was found to range from 1.75 to 4.5. 


SUMMARY 

This study is an analysis of the validity 
and reliability of the Peabody Library Infor- 
mation Test as administered to 1263 fresh- 
men and 59 juniors under ordinary classroom 
conditions in nine colleges, when accepted 
statistical measures of validity and of re- 
liability are applied to it. 

The findings indicate that the test is 
capable of a high degree of discrimination 
between freshmen classes of different schools 
as well as between classes at different scho- 
lastic levels within the same college. It also 
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measures a distinct difference in the amount 
of knowledge possessed by a junior class 
before and after two weeks’ instruction in 
library usage. 

The eight separate sections of the test 
show such low intercorrelations as to indicate 
that each is measuring a separate function 
which is not measured by either of the other 
seven parts. 


In only one group was the reliability ¢ 
efficient found to be below .81, as obtained 
by correlating the even items with the odd by 
the Pearson product moment method ang 
applying the Spearman—Brown  correctip; 
formula. This coefficient of reliability was 
found to be as high as .96 for one group 

The P. E..,, for an individual’s raw » 
ranged from 1.75 to 4.5. 





AN ANALYSIS OF THE RESULTS FROM THE FIRST ANNUAL 
EDITION OF THE NATIONAL TEACHER EXAMINATIONS 


Joun C. FLANAGAN 
Cooperative Test Service 


INTRODUCTION 


[his discussion is a report of the results of 
the first annual administration of the National 
Teacher Examinations. Discussions of the 
nature of the examination service and its 
development? and of the administration of 
these examinations? have been published else- 
where. The National Committee on Teacher 
Examinations was appointed by the Amer- 

in Council on Education at the request of 

group of superintendents in the spring of 

9. The Carnegie Foundation for the Ad- 
vancement of Teaching provided funds for 
the development of the project, and the 
National Committee assigned to the Cooper- 
tive Test Service of the American Council on 
Education the task of preparing the annual 
ised 

The service made available to superin- 
tendents and boards of education by the 

National Committee on Teacher Examina- 
tions has three principal objectives: first, 
he provision and administration of a battery 
f teacher examinations; second, the collation 
ind distribution of information on the ap- 
praisal of other factors to be considered in 
the selection of teachers; third, the promotion 
i research in teacher education and guidance. 

It should be stated at the outset that the 
superintendents who inaugurated this project 
were well aware that no battery of tests could 
at the present time make possible a complete 
evaluation of all factors to be included in the 
selection of teachers. However, this group of 
superintendents felt that having objective and 
comparable information on certain aspects of 
training and professional preparation would 
be of great assistance to them with their in- 
dividual problems of teacher selection. In 
common with all other persons who have been 
confronted with the task of teacher selection, 
these superintendents fully realized the diffi- 


a@ Teacher Examination Service, 
National Committee on Teacher Examinations of the American 
Council on Education, 15 Amsterdam Avenue, New York City, 

apeaner 1939, 
yans, David G. “The Professional Examination of 
Teaching Candidates: A Report of the First Annual Admin- 
rat ory National Teacher Examinations,”’ School and 

273-284. 


of the battery of examinations to be 


‘An Announcement of 


October 5, 1940). pp. 
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culty of interpreting the types of information 
which have previously been available to them. 

The selection of general areas to be tested 
was made chiefly through a series of confer- 
ences with superintendents and a question- 
naire sent to a larger group of superintend- 
ents in the spring of 1939. Information about 
such traits as desirable attitudes and personal 
characteristics had to be eliminated from the 
proposed battery of examinations since satis- 
factory techniques for measuring these traits 
objectively are not available. In general, the 
superintendents stated that they wished in- 
formation about the extent to which candi- 
dates could read with understanding, could 
express themselves clearly, and could handle 
numerical and non-verbal concepts intelli- 
gently. They also wished to know to what 
extent the candidates had acquired a good 
liberal education and the extent of their 
awareness of contemporary affairs and cur- 
rent social problems. They wished informa- 
tion concerning the degree to which the candi- 
date was aware of the social implications of 
education, his knowledge of the basic prin- 
ciples of psychology as they apply to educa- 
tion, and his familiarity with the techniques 
of guiding and assisting students. Lastly, they 
wished to know the extent to which the 
teacher was really a master of those fields in 
which he proposed to teach. 

Starting with this general outline, for which 
these superintendents also supplied approxi- 
mate time allotments, the staff of the Coop- 
erative Test Service began in the summer of 
1939 the preparation of the first battery of 
examinations. 


Over a period of ten years the Coopera- 
tive Test Service has developed a method of 
test construction which has proved quite 
effective. Following this method, each step 
in the construction of the tests is performed 
by those individuals most competent for this 
particular task. The method utilizes four 
groups of individuals: first, editors in the 
various fields in which examinations are be- 
ing constructed who are experts in the subject 
matter of those fields and who have unusual 
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ibility to express ideas clearly and concisely; 
second, scholars, supervisors, and other ex- 
perts and authorities who are selected from 
the entire country and who are responsible 
for the planning and final editing of the vari- 
ous examinations; third, experts in the tech- 
niques of test construction who assist the edi- 
tors and committees with the technical 
pects of the preparation of the examinations; 
fourth, a large group of test users who act as 
test critics and collaborators and who insure 
that the final form of the examination 
practical as well as valid. 

The steps involved in the actual construc- 
tion of these examinations may be briefly 
outlined as follows: 


as- 


1S 


1. Developing outlines and_ specifications 
for the various examinations to be con- 
structed, in cooperation with a group of 
authorities and experts. 
Writing test items in 
the specifications and 
editing the items. 
\dministering these preliminary items 
to students in teacher training institu- 
tions to eliminate weak and ambiguous 
items or items that are too easy or too 
difficult. 


accordance with 
outlines, and 


Selecting and revising the items for the 


tentative final forms of the examina- 
tions. 

Obtaining criticisms and editorial assist- 
ance from a large number of super- 
visors, heads of departments, scholars, 
experts in teacher training, and similar 
individuals. 
Editing on 


ceived. 


basis of criticisms re- 


the 


As mentioned in the outline above the in- 
itial work on the-development of the outline 
specifications for each of the various tests was 
done by the staff editor in the particular field. 
This tentative outline was prepared after 
several weeks of careful research into what 
experts and school officials consider should be 
the relative attention given to the various 
sub-topics which are included under each of 
the major classifications into which teacher 
education is divided. Courses of study, text- 
books, journal articles, and reports by various 
professional organizations were analyzed care- 
fully, and these findings were briefly summar- 
ized for the convenience of the experts and 
authorities who were cooperating in the prepa- 
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ration of the outline for the parti 
These materials were sent out to ab 
dividuals, who were asked to suggest 
in the tentative outlines or in the 


tions tentatively drawn up for the writing » 


the examination questions. An eff 
made to have this group of cooperat 
viduals as representative as possible o! 
interested groups. Because of the 

of time allowed for the preparatio: 
first form, it was impossible to make 
ough a canvass of interested groups 
expected will be made with later for: 
it was considered that a fairly represent 
group was obtained. On the averag: 
were about eleven people who cooperat: 
the preparation of each of the outlines. 7 
total group was divided as follows: 
Teachers’ college officials ........... l 
Teacher training personnel in other col- 

leges and universities 
Instructors in academic fields not prin 

cipally concerned with teacher training 18 
Teachers and supervisors in city public 

school systems . 

In writing the items for the various « 
inations, an effort was made to test 
ability of the candidate to apply to pract 
school situations, or, in some cases, tf 
situations of other varieties, the facts 
principles which he had learned. In 
this, much use was made of the best 
technique; that is, the situation was desc: 
and several possible answers, usuall) 
were provided. 
select the answer which he regarded as | 
The preparation of such items is, of 
very much more difficult than the writing 
items which test the simple recognitio 
facts as they have been taught; but the 
sultant test is generally regarded as a 
more valid indication of the teacher’s 
to show good judgment in an actual teachi 


The candidate was asked t 


1 
hing 


situation. The inclusion of only those items 


upon which the experts are in agreement 
cerning the best answer appears to be 


the 


most satisfactory method available at present 
of providing a test which will be fair to the 
applicants and at the same time enable th 
employing officials to discriminate among 


candidates on this basis. 


The examinations prepared in this way wer 


administered to 
three examining centers on March 
1940. The examination centers participat! 


3726 candidates in twent) 


\- 


no 


in this first administration of the National 
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examinations were carefully selected 
asis of the ability of the school sys- 
ten take full advantage of the examina- 
‘on service. A number of the cities had pre- 
viously been using examinations as part of 
ir teacher selection program, and merely 
stituted the National Teacher Examina- 
‘ions for the tests previously used. After ad- 
ministration the examinations were all re- 
‘yrned to the central office for scoring by 
means of the International Test Scoring ma- 
-hine. The scores were reported to the school 
systems and to the candidates on April 30. 
{ general report of the administration of the 
examinations has been prepared by Dr. David 


G. Rvans.* 


tne 


RESULTS 


The most obvious finding from the admin- 
stration of these examinations is that candi- 
lates for teaching positions are not equally 
well qualified. The distributions of scores are 
shown in Table I. Inspection of the table 
makes it clear that although all of these indi- 
viduals have satisfied the legal requirements 
for certification and have accumulated the 
necessary credits, some of them know so little 
n various fields that they are able to answer 

rrectly practically none of the questions in 

e examinations, whereas others make very 
nearly perfect scores on the examinations. 

\lthough a very large amount of evidence 
has accumulated through other testing pro- 
grams, such as the National College Sopho- 
more Testing Program sponsored by the 
\merican Council on Education and the test- 
ing program of the Teachers College Person- 
nel Association, showing that large differences 
exist. between institutions, it seems worth 

hile to add evidence from this particular 
testing program. The distributions of total 
scores on the common examinations for seven 
institutions from each of which there were 
more than seventy-five students who took the 
National Teacher Examinations are shown 
graphically in Figure 1. The chart shows the 
scores achieved by the students from two state 
universities, three private universities, and 
the teachers’ colleges of two different states. 
It should be emphasized that these charts are 
based on only those students from the various 
institutions who took the teacher examina- 
tions this past spring. Although these groups 
are therefore not known to be representative 
{ the entire student body or even the group 
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of students prepared to be teachers by the 
particular institution, the charts show that of 
the students presenting themselves for exami- 
nation eighty-five per cent of those from one 
institution did better than did the average 
student in two of the other institutions. It is 
quite likely that some of the differences would 
be even larger were records on the entire grad- 
uating groups available for all these colleges. 
However, it should not be overlooked that 
these results indicate that there are some very 
good studies in all of these colleges. 

A second point of considerable interest 
which is brought out by an analysis of the 
results is that @ fairly long and varied battery 
of examinations is necessary to describe the 
various candidates adequately. The first evi- 
dence which will be presented on this point is 
the profiles on the battery of examinations 
shown in Figure 2. This figure is reproduced 
from the profile chart which was sent to each 
individual who took the examinations with the 
suggestion that he plot his own profile on the 
chart in order to interpret more readily his 
own pattern of achievement. The ten profiles 
on the chart, representing the average scores 
made on the various common examinations by 
the group of candidates taking a particular 
subject examination, clearly emphasize the 
marked differences in pattern of these groups, 
even in cases where the combined averages 
are very similar. The profiles are self-explan- 
atory and seem to be about what one would 
expect. The Mathematics and Physical Sci- 
ence groups are outstanding on the Reasoning 
Test, the Language groups are superior on the 
English Expression Test, and the Social 
Studies, Science, and Mathematics groups are 
high on their respective sections in the Gen- 
eral Culture Test. 

Although all the various examinations in 
the battery are positively correlated, the fact 
that each one is making a unique contribution 
to the total battery is shown by the intercor- 
relations of these various parts presented in 
Table II. The smallest of the Pearson 
product-moment correlation coefficients shown 
in the table is .23 between the Fine Arts and 
the Mathematics sections of the General Cul- 
ture examination. The largest coefficient is 
.81 between the Current Social Problems and 
the History and Social Studies sections of that 
examination. The median of the fifty-five cor- 
relation coefficients in the table is .56. Pairs 
of tests which would be expected to show sub- 
stantial correlation, such as Fine Arts and 
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FicureE 1 


‘ 


THe Grapuic DISTRIBUTIONS OF THE TOTAL ScorRE ON ALL THE COMMON EXAMINATIONS IN THI 
TEACHER EXAMINATIONS ACHIEVED BY THE CANDIDATES TAKING THE EXAMINATIONS IN Marcu 
Wuo Hap Recetvep THertrR TRAINING AT CERTAIN SELECTED INSTITUTIONS 


A Private A State T.Cole of T.Cole of A 


Percen-A Private A Private 
one state one state 


tile Unive Univ. Univ. Univ. 


x x x a x 











x x ¥ 
The percentile scale at the left is based on the returns for all the candidates taking the National Teacher 
Examinations. The middle horizontal lines show the national median, and the other two are at the 16th and 
84th percentiles of the distribution. The wide portion of each bar represents the range of scores of the middle 
half of the candidates from the particular institution. The narrow parts extend to the 16th and 84th per- 
centiles. The lines at the end extend down to the 10th percentile and up to the 90th percentile. The crosses 
represent the minimum and maximum scores for each institution. The short horizontal line near the middle 
of each bar represents the median score for the particular institution. 








NATIONAL TEACHER EXAMINATIONS 


FicuRE 2 
INDIVIDUAL PROFILE CHART FOR NATIONAL TEACHER EXAMINATIONS 
May, 1940 
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Drrections. It is suggested that each candidate fill in his VALUE oF THE CHarT. This chart enables the candidate to 
wn test results as reported on the card sent to him May first appraise his own performances in various ways: 
This may be done by drawing circles around the dots cor- 1. He can compare his score on any test with the Scaled 
responding to his scores on the various subtests of the National Score 50, which is the score that the “average American’’ 
Teacher Examination Battery. (1.Q. 100) would be expected to make upon graduating from 
Each of the ten profiles already drawn on the chart repre- an average college. 
sents the average scores made on the various common exam- 
nations by the group of candidates taking a particular special 
subject examination. The code for identifying these special 
subject group profiles is shown below: 

1. Education in the Elementary School— - — - —.- — 

English Language and Literature 
Social Studies 


2. He can compare his score on any test with the median 
scores of candidates taking any particular option by means of 
the profiles shown on the chart. 

3. He can study his own relative achievements in different 
fields by direct visual inspection of his profile, since all tests 
are reported in Scaled Score units. This is perhaps the most 
Mathematics important use of the chart, since it gives the candidate direct 
Biological Sciences aid in appraising his own particular strengths and weaknesses 
Physical Sciences It should be noted that many of the superintendents in the 
French — — larger examining centers are likely to be interested in receiv- 
German ing applications and transcripts from candidates whose scores 
Spanish tend to be “exceptional,” i.e., around 80 or above on the 
Latin various examinations. 
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TABLE II 
INTERCORRELATONS OF SCORES ON THE TESTS OF THE 1940 TEACHEX EXAMINATION BATTERY Osta 
4 SAMPLE OF ONE-FIFTH OF THE CASES SELECTED TO BE REPRESENTATIVE OF ALL CENTERS 
ALL Score Levets (N = 668 
Test 1 2 3 4 5 6 7 8 9 
Reasoning 1 45 45 43 34 .28 .46 32 67 ; 
English Comp 2 73 -75 .68 -76 51 73 44 ‘ 
English Exp. 3 51 45 .67 .29 59 40 
G.f Cur. Soc. P i 81 59 .58 60 A7 8 
G. Hist. & Soc. St 5 64 .49 58 40 
(;.f Literature 6 "26 76 27 
Gat Science rf 37 55 
G. Fine Arts a! 23 
G.f Mathematics Q 
Prof. Information 1 
Cont. Affairs 1 . 
M 60.43 64.37 61.76 62.44 62.98 63.34 60.59 63.52 62.05 625 
1.78 11.30 12.56 11.74 11.95 1.77 11.48 11.55 11.68 ! § 


Literature, Current Social Problems and Con- 
temporary Affairs, and English Comprehen- 
sion and English Expression turn out to do 
so, and other pairs show anticipated lack of 
agreement. The Reasoning Test, which was 
the only non-verbal test in the battery, shows 
very few substantial correlations. It is most 
closely associated with the Mathematics sec- 
tion of the General Culture Test. It is inter- 
esting to observe that the next largest cor- 
relation coefficient for this test is with the 
Professional Information Test. This is grati- 
fying since an effort was made to introduce a 
substantial factor of professional judgment 
into the questions in the latter test. 

It is of some interest to note that the evi- 
dence from these intercorrelations and from 
the profiles for the subject-matter groups 
seems to suggest three main factors included 
in the various tests. Practically all the tests 
ind especially those in the fields of English, 
Foreign Languages, Social Studies, Fine Arts, 
ind Professional Information show high cor- 
relations with English Comprehension. This 
general agreement of tests in these fields has 
been observed in many previous studies and 
is usually considered to be due to a general 
verbal or linguistic factor which is important 
in practically all types of academic work. A 
second general factor which is indicated by 
the results appears to include mathematics, 
science, and non-verbal reasoning. Achieve- 
ment in these fields seems to require some- 
thing more than verbal ability and this some- 
thing more is usually called a quantitative or 
reasoning factor. A third factor which is not 
quite so clear-cut as the two previously men- 
tioned is related to the Current Social Prob- 
lems, Contemporary Affairs, and Social 
Studies tests. This factor, consisting princi- 
pally as it does of an awareness of social 


problems and current happenings, 
what some people have called “si 
tivity.” 

Although from inspection it appea 
these three factors should account for 
the agreement in the scores on these tests, jt 
should also be noted that there appears to | 
a moderately large factor which is unique or 
specific to each of the tests. Thes 
factors would be lost to a very large extent if 
the examination were greatly shortened or 
simplified. A more thorough mathemat 
analysis of the test results is planned and will 
be published in a later report. 

One other type of evidence whi 
separate testing and reporting for th 
fields is obtained from a study of 
cases. In Figure 3 are shown the p 
the scores made by two applicants for 
tion teaching the Social Studies in o1 
cooperating cities. One of these candidates 
has a very good record on the History 
Social Studies section of the Genera! ( 
Test and on three of the other secti 
test but is only slightly above the a 
all the Social Studies applicants on 
rent Social Problems part. The other 
date has a lower score on History and 
Studies and certain of the other sections 
the General Culture Test but is outstanding 
on the Current Social Problems sectio 
Other differences in the profiles of these 
didates are apparent. One is much mor 
standing on the Professional Information | 
than is the other. These cases have | 


selected to show that although in 
there is substantial agreement between the 


scores achieved on various tests such as the 
_— 


Current Social Problems and the Histor: 
Social Studies tests, there are individ! 
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FicuRE 3 
FILES OF TWO CANDIDATES FOR A PosITION TEACHING THE SOCIAL STUDIES IN AN EASTERN CITY 
Optional 
General Culture Tests 
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ceptions some of which are quite striking in 
character. 

Numerous examples of cases of this kind 
might be cited, but it should be evident from 
the intercorrelations and other data presented 
that the task of filling a particular position is 
greatly simplified if such a comprehensive 
objective profile is available in addition to 
ratings of training and experience, personality 
characteristics, and other factors considered 
important by the local group. 

There is another point in favor of a rather 
comprehensive examination that should be 
mentioned. Even if it developed that the 
forty-minute Reading Comprehension Test 
ranked the candidates in almost the same 
order as the total battery for this initial 
group, it would be very dangerous to abandon 
the use of the other tests. If it were known 
in advance that the test contained nothing 
but Reading Comprehension, there would be 
grave danger that other factors in the train- 
ing of the candidates would be slighted, so 
that in later groups the Reading Comprehen- 
sion Test would no longer provide the accu- 
rate indication of broad preparation which it 
originally did. For these reasons the National 
Committee plans to keep the battery of ex- 
aminations as comprehensive as possible and 
has taken every possible opportunity to urge 
superintendents and boards of education to 
supplement the examinations so as to include 
as many of the important characteristics of 
good teachers as possible in the selection plan. 

In addition to demonstrating that there are 
very large individual differences in the per- 
formance of candidates on these tests and also 
that the patterns of individuals whose total 
scores are at nearly the same level are fre- 
quently quite different, it seems important to 
present evidence showing as clearly as pos- 
sible what these tésts are measuring. The ex- 
periences of two of the cooperating super- 
intendents will provide a type of evidence 
which should accumulate quite rapidly as the 
tests become more widely used. The first case 
concerns the report to us by the superintend- 
ent of a Western school system that an indi- 
vidual who had taken the examinations this 
March in an Eastern center had written to 
him applying for a position. The superintend- 
ent wrote to the National Committee office 
regarding the candidate’s letter of application 
saying, “His use of the English language in 
form, in spelling, and in grammar was such 
that we had to tell him that we were not in- 
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terested in any candidate of his type.” Ip. 
spection of the records in the central! Office 
revealed that this particular candidate rankes 
close to the bottom among all candidates 
taking the examinations. He was in the low. 
est seventh of the distribution in both English 
Comprehension and English Expression. 

The superintendent in another city where 
the examinations were used this past spring 
talked with a number of candidates who hag 
taken the examinations and were applying for 
positions in his system. He reported to the 
National Committee that he found his judg. 
ment of these candidates substantially 
agreement with the results which they 
tained from the National Teacher Examina- 
tions. The National Committee plans to make 
a systematic survey this fall to discover the 
relationships between various factors usually 
associated with successful teaching and th 
scores achieved on the examinations. It js 
hoped that this survey will bring out more 
clearly those elements of successful teaching 
not adequately covered by these examinations 
so that the examinations may be extended 
and so that school officials may be assisted in 
appraising those factors for which objective 
examinations are not now available. 

In the meantime it is believed that the best 
evidence concerning the validity of the exam- 
inations is contained in the items themselves 
Therefore, a number of items are presented 
below to enable the reader to judge for him- 
self as to whether the examinations are pro- 
viding valid measures of factors concerning 
which superintendents desire information 
These particular items have been selected for 
presentation here because they provide the 
most valid brief indication of the contents of 
the test which is now available. In the case of 
most of these items the chances are about 
even that the average candidate will know the 
answer. But for selected persons obtaining 
scores within the top ten per cent on a par- 
ticular test, the chances are generally about 
ten to one that they will know the answer to 
any of these sample questions from the test. 
These are the items which agree best with the 
scores on the various examinations, and there- 
fore provide the best illustrations of the 
meanings of these scores. 

It is apparent that the average candidate 
applying to a superintendent for a position 
will not usually know the answers to these 
questions. On the other hand, if superintend- 
ents or boards of education wish to emplo) 
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»dividuals who can answer such questions as 
these, they may insure this by selecting those 
-andidates who in addition to having the other 
iesi rable qualities for the position, obtained 
high scores on these examinations. In looking 
er these questions from the common exam- 
ations required of all candidates the reader 
hould remember that, although the items pre- 
ented | below include a very limited sample of 
he test, they illustrate the type of question 
whi h is known to the high-scoring individ- 
iis but is as likely as not to be unknown to 
iverage applicant for a teaching position. 

t should also be remembered that, as indi- 
ited in Table I, there are quite a number of 
pplicants for teaching positions in the city 
S cho ol systems included this year who know 
» answers to Aardly any of these questions. 
(hese individuals, all of whom have fulfilled 
necessary requirements for teaching cer- 
tes and many of whom have actually 
tained teaching positions, get a few items 
partly by chance, but when their 
they are 


rrect, 

res are corrected for guessing, 
ery close to zero. 

In general the directions for these items 
isk for the selection of that one of the five 
hoices given which best completes the state- 
ment. In a few cases the candidate is asked 

find the one which is wrong. 


LIsh COMPREHENSION, Part II, 
It must not be supposed that the officials in charge 
of education desire the young to become educated. 
ontrary, their problem is to impart infor- 
ithout imparting intelligence. Education 
have two objects: first, to give definite 
viedge; second, and more important, to create 
mental habits which will enable people to 
knowledge and form sound judgments for 
lves. The first of these we may call infor- 
mation, the second intelligence. The utility of in- 
telligence is admitted only theoretically, not prac- 
tically: it is not desired that ordinary people 
ild think for themselves, because it is felt that 
eople who think for themselves cause administra- 

tive difficulties. 


On th 


According to the writer, a truly educated 
population is regarded by the officials in charge 


theoretically undesirable. (23%, 2%)* 

the hope of democratic society. (4% 

3 the main objective of the scbeals. 

1%) 

unnecessary. (5% 

hard to manage. 705%, 94% ) 

omitted (0%, 0%) not reached (0%, 0% ) 
* After each answer the percentage of all candidates select- 

ng that particular choice is given; the second figure (in 

italics) is the percentage of those individuals with scores in 


the highest ten per cent who chose that answer. The correct 
answers are printed in bold-face type. 


wwe 


% 
rt %, 


nn 
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6. The writer evidently considers that the most 
important function of education is to 
1 impart useful information. (279%, 870} 
2 provide vocational training. (0%, 0%) 
3 develop intelligent thinking. (69%, 
91%) 
4 create a stable social group. (3%, 1%) 
5 develop well-adjusted personalities. (1%, 
%) 
omitted (0%,0%) not reached (0%, 0%) 
Slade was a matchless marksman with a navy re- 
volver. One morning at Rocky Ridge when he was 
feeling comfortable he saw a man approaching 
who had offended him some days before. “Gentle- 
men,” said Slade, drawing, “it is a good twenty- 
yard shot—I'll clip the third button on his coat!” 
Which he did. The bystanders admired it. And 
they all attended the funeral, too. 


- 


7. Slade may best be described as 
1 humorous. (3%, 0%) 
2 cold-blooded. (44%, 
3 revengeful. 4%, 19%) 
4 playful. (8%,7%) 
5 daring. (11%, 3%) 
omitted (0%,19%) not reached (0%, 0%) 
8. The passage achieves its effect chiefly by means 
of 


70%) 


1 suspense. (13%, 10%) 

2 exaggeration. (5%, 0%) 

3 direct quotation. (20%, 3%) 

4 matter-of-fact treatment. (60%, 
86% ) 

5 detailed description. (2%, 0%) 
omitted (0%,1%) not Sentai a (0%, 0%) 


ENGLISH COMPREHENSION, Part III, VOCABULARY, 


84. pre ponderant 

1 determined. (5%, 2%) 

2 preparatory. (0%, 0%) 
advanced. (0%. 1%) 

4 thought out. (21%, 5%) 

5 predominant. (30%, 90% ) 
omitted (16%,2%) not 
0%) 


reached (28%, 


Encuisn Expression, GRAMMATICAL USAGE, 


3 a magnificent hotel; 


37. It was truly 
1 (4%. 0%) 
every one of the washstands in the various 
guest rooms were of marble, 
2 (64%, 96%) 
as were the me ace mantels. 
3 (10%, 0 4 (0%, 0%) 
5 none te wrong (18%, 4%) 
omitted (4%,0%) not reached (0%, 0%) 


ENGLISH EXPRESSION, PUNCTUATION, 
All the childrens art classes were represented in 


/ 
the exhibit, and the class which had done the best 
8 
work received a prize. 


7. 1 No punctuation necessary. (10%, 1%) 
2 children’s. (56%, 84%) 
3 childrens’. (27%, 15%) 
omitted (5%,0%) not reached (2%, 0%) 
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8. 1 No (73%, 
95%) 

class, (18%, 4%) 
omitted (7%, 1%) 
No. punctuation 
94%) 

work, (24%, 4%) 
omitted (9%, 2%) 


punctuation mecessary. 


not reached (2%, 0% 


mecessary. (65%, 


not reached (2%,0%) 


ENGLISH EXPRESSION, CAPITALIZATION, 
“The truth is,’”’ she 
to mail them.” 14 
14. 1 capital letter (19%, 8% 

2 small letter. (67%, 92% ) 


omitted (4 %) not reached 


continued, “we aln forgot 


(10% 


SION, SPELLING, 


(58%, 94%) 


ENGLISH EXPRE 


12. 1 potatoe. 


tty 


URRENT S¢ 


nteed to the 


| code (O 


3 first amenc P hae to the Const tution 
(63%, 98% ) 
Zenger decision. (2 

ition of Independence. (2 

itched (0%, 0% 


5 Declar 
omitted (1% %) not re 
te has great power In labor 
ontrol over 
ibor relations 1 wf, %) 
enditure of relief funds for strike 


( suance injunctions. (10%, 7% 
4 the National Guard. (56%. 91%) 
terstate commerce laws 10% N 
mitted (1%,0%) not reached (11%. 0%) 
ist fifty vears, man the Western 
I rained most in 
1 health. (52%,97°% ) 
native intelligence. (6% 
emotional stability. (4° 
of employmen 
of government 
1%,.0%) not 


0% 
0, O% 

(6%. 1%) 

(14%. 2% 
reached (18 


urity 
, 


or 


Part II, History Anp Soctar 


DIES 
An innovation of the last ten years has been 
he United States government’s encouragement 


1 collective bargaining. (52%, 93%) 

2 the regulation of interstate commerce 
(14%, 1% 
the policy of 
positions on the basis of merit. (8%, 

4 cooperative action to reduce prices. 


appointment to government 
0%) 


(6% ©, 


cr 5c" 
t - c 

(2% 
- c 


, 0%) 


>) not reached 


OF EXPERIMENTAL EDU( 


‘ATION 


58. Neutrality is most often endangered 


GENERAL 


(;ENERAL (¢ 


). The 


GENERAL 
42 


putes arising in connection with 


1 the transport of soldiers. (2%, 1 
2 radio propaganda. (1%, 0%) 


; 
3 seaborne commerce. (57%, 92 
4 


interference in the internal policy 
other nation. (19%, 3%) 

5 the public utterances of persons 
position. (4%, 4%) 
omitted (6%, 0%) not reached 


Part III, Literatur: 
One of the greatest of French essayist 

1 Ronsard. (2%, 0%) 

2 Montaigne. (54%, 98°) 

3 Moliére. (13%, 2%) 

4 De Maupassant. (16%, 0% 

5 Villon. (1%, 0%) 

omitted (13%, 0%) not reached 


CULTURE, 


who “sounded his bart 
roofs of the world.” vet 
new freedom and flexibility to 
1 T. S. Eliot. (4%, 4% 
Emerson. (6%, 0% 
3 Edgar Lee Masters. (4%. 
Edwin Markham. (4%, 0%) 
5 Walt Whitman. (55%, 95° 


nitted (21%,1%) not reached 


The poet 
over the 


Part IV, Scren 
the following 
moonr 


“ULTURI 
W! ch one ol 
affected by the 
1 Radio communication. (2° 
Proper planting periods 
Fertility of animals. (2% 
Probability of rain. (12 
Sea level (62%, 100%) 
omitted (6%,0%) not reached 
The feature which 
man’s nervous system from that of 
tebrates is the greater development 
th 
1 respiratory center 
» sheathing (470, 
organs. (10%, 1" 
+ spinal ganglia 2 } 
5 forebrain. “(67° 0, 94%) 


omitted (5%,09) not reached 


2 nervy 
sensory 


2% 


/ 


physical discomfort produced by 
badly ventilated room full of seal 
mainly to 
1 lack of sufficient 
cumulated carbon 
vapors 


oxvgen (24% 

dioxide 

3) poisonous from 
(4%, 1% 

4 accumulated water 
68% ) 

5 the very 
3%) 


omitted (0%, 0%) 


pec yp { 
vapor. 


low humidity of the ar 


not reached (8 


Cutture, Part V, Fine Arts, 
The best architecture is generally con 
be that in which beauty is the natura! 
sion of 

1 national feeling. (12%,1%) 

2 function. (48%, 91%) 


especially distil 


(28° 
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style of the period. (13%, 1%) 
strength. (13%, 6%) 
1 fine sense of decoration. (3%, 1%) 
mitted (11%,09%) not reached (0%, 0% 


t modern painters are | 
the revival of mural painting today ? 
English. (5%, 07) 
German. (2%, 2%) 
Italian. (12%, 3%) 
Mexican. (53%, 91%) 
French. (8%, 1%) 
mitted (169%,3%) not reached (4%, 0% 


ULTURE, Part VJ, MATHEMATICS, 
one of the following equals 0.25%? 
0025. (53%, 95%) 
25. (6%, 0%) 
25. (34%, 5%) 
? 50. (0%, 0% 
5.00. (4%, 0% 
mitted (3%,0%) not reached ( 
ositive square root of % is 
14. (54%, 100%) 


0%) 


tted (12% 


iL INFORMATION, Part I, EDUCATION AND 
Po.Licy, 

irt the school is permitted to play in the 
pment of society is determined chiefly 


attitude of the society which main- 
tains it. (67%, 87%) 

moral courage of the teachers. (1%, 0% 
ability of the teachers to construct a pro 
gram for social change. (19%,7%) 
power of judgment developed in the chil 
dren. (11%, 5%) 

willingness of the school to follow political 
domination. (0%, 0%) 

omitted (2%,1%) not reached (0%, 0% 


stantial modifications in the high school 
rriculum have been necessary since the World 
to allow for 


expansion of the college preparatory cur- 
riculum. (14%, 5%) 

the presence in high school of many 
persons with slight aptitude for 
book learning. (61%, 88%) 

3 studies on a more difficult level, necessi- 
tated by better preparation in the grades 
(3%, 3%) 

emphasis on cultural rather than voca- 
tional subjects because of decreased em- 
ployment opportunities. (5%, 2%) 

5 consolidation of rural high schools. (10%, 
2%) 

omitted (7%,0%) mot reached (0%,0% 


ely responsible 


PROFESSIONAL INFORMATION, Part II, Cuitp Devet- 


OPMENT AND EDUCATIONAL PSYCHOLOGY, 
19. If a child is continually a difficult behavior 
problem, the teacher might best 


1 refer his case to a competent author- 
ity for investigation. (71%, 90%) 

2 discuss the problem with other teachers 
(9%, 5%) 

3 isolate him from the rest of the class 
(2%, 0%) 

4 have him transferred to another class 


Ov" 


5 devise a more effective form of discipline 
16%. 5% 
omitted (1%,0%) not reached (0%,0%) 


55. Six-year-old children differ most from ten- 
vear-olds in which one of the following re- 


pec ts? 


1 Fine-muscle control. (48%, 80% ) 
2 Large-muscle control. (8%, 3% 
Visual acuity. (3%, 2% 
4 Auditory acuity. (1%. 0 
5 Metabolic rate. (20%, 13%) 

omitted (14%, 2%) not reached (6%. 0% 


or 


FESSIONAL INFORMATION, PART III, GUIDANCE AND 
INDIVIDUAL AND Group ANALYSIS, 


18. A boy applying for entrance to a highly sele 
tive school attained a percentile score equiva 
lent to the 67th percentile of the previous 
year’s entering class in that school. The school 
counselor is justified in concluding that the boy 


1 probably should not be admitted to the 
school. (8%, 8%) 
if admitted, would probably fail in his 
work. (2%, 0% 

admitted, would have to work harder 
than the average entering student in order 
to avoid failure. (30%, 179%) 

4 if admitted, would probably do 
slightly better than average work. 
(36%, 70%) 

5 if admitted, should be about 67th best in 
the entering class of one thousand. (1( 
omitted (14%, 390) not reached (0%. 0% 

26. It is a basic assumption of experts in voca 
tional guidance that 

1 interest in an occupation is the best evi 
dence of future success in that occupation 
(10%, 0%) 

2 vocational aptitudes cannot be measured 
objectively. (4%, 2%) 

3 pupils need assistance in estimating 
their own abilities, aptitudes, and 
interests. (84%, 98%) 

4 there is one, and only one, occupation for 
which each individual is ideally suited 
(0%, 0%) 

5 vocational and _ educational guidance 
should be kept entirely separate. (1%, 0%) 
omitted (1%,0%) not reached (0%, 0%) 
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PROFESSIONAL INFORMATION, Part IV, SECONDARY 
ScHOOL METHODS, 

1. The primary function of the recitation should 
be to 

1 give individual attention to the slower 
pupils. (1%, 0%) 

2 discover the pupils who have not assimi- 
lated previous lessons. (6%, 2%) 

3 discover the extent to which the individ- 
ual pupils are able to recall material read. 
(1%, 0% 
develop understanding through 
group discussion. (74%, 90%) 
improve the ability of the pupils to pre- 
sent material orally. (17%,7%) 
omitted (1%,19) not reached (0%, 0%) 

successful, the “no-failure” policy 

adopted in some schools requires 
1 the individualization of programs of 
study. (60%, 80%) 
2 decreasing attention to the superior stu- 
dents. (4%, 3%) 
3 the abandonment of school grades. (20%, 


12¢ 


38. To be 


de 


emphasis on vocational education. (4%, 


elimination of pupils who cannot do sat- 
isfactory work. (4%, 2%) 
omitted (89,19) not reached (0%,0%) 


SUMMARY AND CONCLUSIONS 


The above discussion gives a brief analysis 
of the results of the first edition of the 
National Teacher Examinations. Those ex- 
aminations were prepared under the direction 
of the staff of the Cooperative Test Service 
f the American Council on Education with 
the very generous cooperation and assistance 
of more than two hundred experts from all 
parts of the country. The examinations were 
designed to include those aspects of the candi- 
dates’ training and ability about which the 
uperintendents desired information and _ for 
which satisfactory techniques of objective 
measurement were available. 

Only those items were included in the ex- 
amination which performed satisfactorily in a 
preliminary tryout and which were selected 
and agreed to by a large number of experts 
and critics including subject-matter scholars, 
teacher training experts, and selected teachers 
and supervisors in city school systems. An 
effort was made to make the examinations as 
functional and valid as possible by stressing 
the application of facts and principles and 
the candidates’ ability to exercise good judg- 
ment in selecting the best of a number of 
possible solutions of practical school problems. 

The analysis of the results indicates that 
candidates for teaching positions are not 


equally well qualified; and that a jair 
and varied battery of examinations is nm 
sary to describe the various candidat, 
quately. And finally, evidence is presented ; 
illustrate the validity of the various parts 
the examinations in measuring their respectixe 
aspects of teacher preparation. 

In conclusion a word should be said at 
the use of these examination results 
National Committee established 
centers in about twenty of the larger cit 


for the initial administration of the tests jp 
March, 1940. The National Committee has 
been interested to observe that in each of ¢! 
more than twenty school systems requiring 
candidates to present transcripts of their rec- 
ords on the National Teacher Examinations 
the selection procedure has also included an 
appraisal of training, experience, recommen- 
dations, and the more intangible personality 
characteristics. A further point of interest 
that no two of the cities have adopted th 
same procedure. Some give the examinations 
a weight of about one-third, others one-half 
some use the examinations as a preliminary 
qualifying hurdle, others require them only 
persons who have been tentatively selected 
the basis of preliminary interviews and 
ings. But all of them require an interview 
and all seem to be making every effort to ¢ 
as much information as possible from perso 
with whom the candidate has studied 
worked. A number of superintendents 
stated that the availability of these ob} 
and comparable examination records ! 
them of the burden of interpreting ambigu 
records of scholastic achievement and « 
them to focus their attention more clear!) 
the appraisal of personality characte! 

The National Committee on Teacher 
aminations and the staff of the Cooper 
Test Service wish to take this opportunity ' 
acknowledge the very kind and generous 
operation of the many individuals in colleges 
and in public school systems who assisted in 
the preparation and administration of thes 
examinations. Suggestions concerning the ex- 
aminations and services provided by the 
National Committee are welcomed. It 's 
hoped that, through the active support o! the 
many persons interested in this project, 
will be possible to make further improvements 
in the examinations and establish a larger 
number of examining centers for the con- 
venience of candidates in the coming year. 














[he prediction of scholastic success in 
-arious school subjects at the secondary and 
lege level is one of our major personnel 
roblems. Vocational guidance during this 
period consists quite largely of counseling 
with regard to the subjects which the student 
ist take in order to prepare for entrance to 
ertain occupation. The student’s previous 
scholastic record suggests itself as one of the 
best predictive indices of success in various 
curricula or subjects, but there are many 
ccasions when the counselor has need for 
special measuring devices to supplement the 
student’s record of scholastic attainments. 
Questions of differential prediction cannot be 
snswered satisfactorily by an examination of 
school records alone. 

\ related personnel problem is that of 
idjusting instructional techniques to the stu- 
dent’s particular level of ability. Granted 
that the student must take a certain subject, 

is still possible to make adjustments in 

biect matter and techniques of teaching 
vhich will enable him to achieve satisfac- 
rily. A student who under ordinary circum- 
nces would be doomed to academic failure 
ght conceivably do a satisfactory grade of 
work if his level of ability were to be taken 
to consideration. Likewise the student with 
high aptitude might gain a great deal more 
1 course if the subject matter proved 
sufficiently challenging. Hence, for both edu- 
itional and vocational guidance purposes, 
iptitude tests for specific subjects or groups 
subjects should prove valuable in clinical 
inseling. 

(he purpose of the present study was to 
identify certain fundamental factors related 
to scholastic success in college physics. For 
1 number of years the Physics Aptitude Test 
of the Iowa Placement Examinations has 
been used at the University of Iowa to pre- 
dict the scholastic success of students enrolled 
in physics courses. It has proved itself to be 
a satisfactory instrument for this purpose, 
particularly in the sectioning of students 
according to ability. Among the other tests 
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measuring factors which seemingly were re- 
lated to physics achievement, it occurred to 
the present writers that tests measuring the 
spatial relations factor might provide valuable 
data which would supplement the information 
provided by the Physics Aptitude Test. Three 
tests already available appeared to measure 
some of these abilities thought to be related 
to physics achievement, namely, the Minne- 
sota Paper Form Board, the Mann Engineer- 
ing Drawing Aptitude Test,’ and the Mann 
Mutilated Cubes Test. Inasmuch as_ the 
majority of these students had taken the Iowa 
Qualifying Examination,’ which includes the 
lowa Mathematics Aptitude Test, the latter 
was included in the experimental test battery. 


Since it was administratively difficult to 
require the entire experimental group of 155 
students to take all the tests, it was broken 
up into two sections, Group I consisting of 
8o students and Group II of 75. Both groups 
were given the Iowa Physics Aptitude Test 
and the Minnesota Paper Form Board. In 
iddition to this battery, Group I was given 
the Mann Engineering Drawing Aptitude 
Test and Group II took the Mann Mutilated 
Cubes Test as the third component of the 
battery. The Iowa Mathematics Aptitude 
Test was available only for those students 
who had entered the University as freshmen 
and therefore a third group of 107 students 
was set up on the basis of these available 
tests. 


During the first semester, the Cooperative 
Physics Achievement Tests (covering Me- 
chanics, Heat, and Sound) were administered 
to all the students, and the total score, con- 
sisting of the sum of the scores on the sub- 
tests, was used as the criterion of achieve- 
ment in physics. Prior to analyzing the pre- 
dictive values of the various batteries, the 

‘Only parts 1, 3, and 7 of this test were used. The re- 
naining parts do not deal with spatial relationships 

2 The score in this examination consists of the sum of the 


we ghted raw scores in the lowa High School Content Exam- 
nation, Iowa English Training Test, Iowa Mathematics Apti- 


tude Test, and Iowa Silent Reading Test. The average cor- 
relation of this examination with first semester grade point 
averages is .65 
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two groups were compared on the basis of 
general ability, as measured by the Iowa 
(Qualifying Examination Composite Score, on 
physics aptitude as measured by the Physics 
\ptitude Test, and on physics achievement 
as measured by the Cooperative Physics 
rests. No statistically significant differences 
were obtained. 

The zero order inter-correlations, computed 
by the product-moment method, between the 
various aptitude tests and the criterion score 
for Group I are presented in Table I 


TABLE I 


ZERO ORDER CORRELATIONS BETWEEN APTITUDE 
MEASURES AND CRITERION FOR Group I 


N 80 
510 r 662 
HOD en 685 
r 258 Yong .160 
1. Cooperative Tests. Physics Aptitude. 
Mann Engineering 4. Minnesota Paper 
Form Board. 


Drawing 


Both the Mann Engineering Drawing Apti- 
tude Test, r .510, and the Physics Apti- 
tude Test, r 505, tend to correlate fairly 
high with the criterion score, whereas the 
magnitude of the correlation for the Minne- 
sota Paper Form Board, r,, .258, is con- 
siderably lower. The character of the other 
intercorrelations is worthy of note, and tends 
to suggest the existence of several factors 
underlying ability in physics. Whereas both 
the Physics Aptitude Test, r 662, and 
the Minnesota Paper Form Board Test. 
- .535, show a considerable degree of 
co-variability with the Mann Engineering 
Drawing Aptitude Test, the correlation of 
the Physics Aptitude Test with the Minne- 
sota Paper Form. Board tends to be quite 
small, r 160. This may indicate that 
although the Minnesota Paper Form Board 
ind the Mann Engineering Drawing both 
presumably measure ability to deal with 
spatial relations, the Mann _ Engineering 
Drawing test tends to be a better measure 
of those components related to physics apti- 
tude and achievement. 


Multiple correlations’ were computed for 
the different combinations of aptitude tests 
correlated with the criterion score and these 

‘These were computed by use of Griffin's technique. See 


Griffin, Harold D.: ‘Simplified schemas for multinle linear 


1 Journ Experimental Education, I (March 


1 pp. 230~254 


are presented below in descending order 
magnitude of R. 












Ri 2% = .588 1. Criterion Score. 

Rie =.557 2. Mann Engineering Drawing 
ti. =.536 3. Iowa Physics Aptituc 

ti: =.510 4. Minnesota Paper Form Board 


For Group I, the best multiple correlatio; 
was obtained for the combination of Man, 
Engineering Drawing Aptitude, Iowa Physics 
Aptitude and Minnesota Paper Form Board 
This result seems in agreement with the pr 
vious inferences deduced from Table | 
cerning the apparently different component 
measured by the Mann Engineering Drawing 
and Physics Aptitude Tests. However, the 
engineering drawing and form board test 
not add greatly to the predictive power 
the physics aptitude test used alone 

As previously noted, Group IT was giver 
the Mann Mutilated Cubes Test instead 
the Mann Engineering Drawing, and i: 
tion, the Physics Aptitude Test and the Mir 
nesota Paper Form Board. The same typ 
correlational analysis used for Groun | y 
repeated for Group IT, the results be 
marized in Table IT. 


TABLE II 


ZERO ORDER CORRELATIONS BETWEEN APTIT! 
MEASURES AND CRITERION FOR GRoUP II 


N = 75 
Y 49 Y's i14 
! 664 Yo = 27: 
ria = .261 Ns = .499 


3. Physics Aptitud 
4. Minnesota Pape: 
Form Board 


1. Cooperative Tests. 
2. Mann Mutilated 
Cubes. 


It may be noted that even under slight) 
different conditions of testing, the correlatio: 
between the Minnesota Paper Form 6 
for Group I, r,, .261, is of comparal 
magnitude to the correlation between thes 
two components for Group I, r,, 258. Tl 
Mann Mutilated Cubes Test does not seen 
to show as high a degree of co-variability with 
either the criterion score or the Physics Apti- 
tude Test as did the Mann Engineering 
Drawing Aptitude Test. The correlation be- 
tween the Physics Aptitude test scores 
Minnesota Paper Form Board is consider 
higher for Group II than for Group I. How- 
ever, by the use of Fisher’s Z-function the 
difference was found not to be significant at 
the 5% level. The same is true of the cor 


hi 
wiv 
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between the Physics Aptitude test hypothesis that these two tests are measuring 
nd performance in the Cooperative very nearly the same thing. 
\chievement test. Achievement test scores for semesters I 
multiple correlations for the different and II were correlated for the two groups in 
tions were determined and are pre- order to determine the consistency of per- 
low in descending order of magni- formance from semester to semester. For 
Group I the correlation coefficient is .774, for 
Criterion Score. Group II it is .913, and for Group III it is 
Mann Mutilated Cubes Test. -782, indicating a high degree of consistency 
Iowa Physics Aptitude. of performance throughout the year. 
Minnesota Paper Form Board. During the year following the completion 
of the research described in the preceding 
paragraphs a supplementary study was made 
of the Minnesota Paper Form Board and the 
Mechanical Movements test of Thurstone’s 
Primary Mental Abilities Test battery. One 
hundred ninety students participated in the 
study. The correlation between the form 
board scores and Cooperative Physics 
Achievement Test was found to be .152, thus 
confirming the findings of the previous study. 
The correlation for the Mechanical Move- 
ments test was .324. This was not as high as 
the correlations for the sub-tests of the 
Physics Aptitude Test as determined in an- 
other study, and consequently the test was 
not regarded by the present writers as meas- 
uring an important component of physics 
aptitude. 


range of multiple correlations for 
II is much wider than in Group I, 
ng that the combination of the Mann 
ted Cubes Test and Minnesota Paper 
Board is not as good a measure of the 
factors related to physics achievement 
Mann Engineering Drawing Aptitude 
\linnesota Paper Form Board. However, 
hysics aptitude test functioned more 
ely as a predictive index for Group II 
r Group I, and as a result all the 
correlations involving it are higher 
negative multiple regression coeffi- 
sulted in a lower multiple correlation 
nt for four variables than for three 
third group was made up of students 
ups I and II who had taken all the 
itive Physics Achievement Tests, and 
m there were available test data on 
wa Mathematics Aptitude and the 
\ptitude tests. The zero order cor- For the population included in this study 
s are presented in Table III. or for similar populations, the following con- 
clusions appear tenable: 

TABLE III 1. Ability in mathematics appears to be 
more closely related than any other factor to 
achievement in college physics. While this 
cannot be regarded as an elemental compo 
nent of physics aptitude, it appears to be 
operationally important. 

2. The spatial relations factor as meas- 
ured by the Minnesota Paper Form Board 
high correlation between Physics Apti- does not correlate highly with success in 
ind Mathematics Aptitude is very sug- college physics. 
ve, and tends to indicate that the two 3. Understanding of mechanical move- 
measure either the same or highly cor- ments as measured by the Thurstone Primary 
related functions. The fact that one of the Mental Abilities Tests does not bear a close 
subtests of the Physics Aptitude Test relationship to success in college physics as 
nsists of problems in mathematics would, measured in this investigation. 
subjective grounds, lead one to expect this 4. The Iowa Physics Aptitude Test and 
result. The multiple correlation, R, ., — .699, the Iowa Mathematics Aptitude Test predict 
is only slightly higher than either r,, | success in college physics with a fairly high 
tends to lend further support to the degree of accuracy. 


CONCLUSIONS AND INTERPRETATIONS 


(DER CORRELATiONS BETWEEN APTITUDE 
SURES AND CRITERION FOR Group III 


N = 107 
661 1. Cooperative Test Score. 
7 2. Physics Aptitude Score. 
3. Mathematics Aptitude Score. 
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FOREWORD 


One of the major problems confronting the 
research worker is that of analyzing collected 
data in preparation for final interpretation 
and presentation. The method to be used for 
the analysis of the data for any type of re- 
search project is an individual question to be 
determined by the type of problem, the ex- 
tent and form of the material at hand, the 
type of analysis to be made and the facilities 
and time available for making the analysis. 

Because of the advantages in speed, accu- 
racy and flexibility of tabulating machine 
analysis in statistical research, it is impor- 
tant that the research worker understand 
some of the fundamental principles involved 
in order that he may determine whether or 
not this method should be used, and if so, 
what preliminary steps should be taken to 
coordinate the collection of data with the 
method of analysis. Tabulating cards are 
physically movable records capable of being 
mechanically arranged, rearranged, counted, 
or tabulated, or a combination of these. Un- 
doubtedly, the most satisfactory means of 
determining the method to be used is to con- 
sult an expert in the field of research project 
analysis, one who can evaluate the various 
methods of analysis in terms of the factors 
governing the method. If such an expert is 
not available, the research worker must make 
his own decision and do his own planning, 
and it is for the purpose of helping him with 
these problems that the following article is 
presented. 

lHE TABULATING CARD 

Ihe tabulating card in most common use 
measures 73@ inches by 3'4 inches and con- 
tains eighty columns of twelve positions each 
(Illustration 1). Ten of these positions are 
the printed digits o to 9 which correspond to 
the digits of numerical data to be punched. 
Iwo unprinted positions are at the top of the 
card and are known as the eleventh and 
twelfth positions, commonly referred to as 


X and R. Data are recorded in this car 
means of punched holes. 

For any project the card must be divided 
into fields to meet the requirements of that 
project. A field is a section of the card con- 
taining an assigned number of columns where- 
in a particular type of information is always 
to appear. For example, in Figure 2 (Exam. 
ple 1), columns 41-43 are reserved as the 
field for pupil number, columns 44-45 ar 
reserved as the field for score. Likewis 
grade, age, and sex are assigned to their re- 
spective fields as shown. Any field must | 
tain enough columns to accommodate the 
maximum number of digits that will be re- 
corded in that field. Fields should be arranged 
so that punching can be accomplished with 
a minimum of effort and time. If the collec- 
tion and recording of data has been properly 
planned, the arrangement of fields on th 
card will be identical with the arrangement 
of the data from which the cards are to | 
punched. 

Any column of the card may contair 
than one punched hole provided that 
sorting and counting operations are to be pe 
formed on that column, or if it is desired t 
record alphabetical characters. Multipk 
punched columns will print from a tab 
only the lowest of the positions punched, ex 
cept where provision is made for printing 
alphabetical characters. Additions can be pt 
formed only from the lowest positions 
punched. The X and R positions are use! 
only for sorting, counting and selection pur- 
poses, except when used in combination with 
another position to form an alphabetical 
character. They can be used neither fo! 
addition, nor indication on a tabulator. 


CoDING 


Unless it is necessary to print a record 0! 
names or descriptions, all non-numerical in- 
formation should be reduced to a numerical 
code. Each type of information, represented 
by the various fields on the tabulating card, 


54 








-hould be coded independently. In many 
sroiects data sheets may be constructed in 
“ych a manner that coding is largely or en- 
tirely automatic, insuring greater accuracy 
ind saving of time for both the research 
rker and the punch operator. 
Fractions should be avoided because there 
means of recording them on a tabulat- 
.¢ card. All fractional parts should be re- 
rded as decimal fractions. 
Measurements to be algebraically manipu- 
lated should not be recorded in compound 
units, but reduced to a single unit. For ex- 
ample, feet and inches should be recorded as 
inches or as feet and tenths of feet. Age 
should be recorded in months rather than 
years and months. 


\W 


If information from various sources must 
be transcribed to a final data sheet, the num- 
ber of digits recorded for each variable should 
be the same as the number of columns re- 
served for that variable on the tabulating 
card. For example, a value of 96 for a vari- 
able that may have a maximum of three 
digits should be recorded as 096, or if four 
digits are a maximum, one being reserved for 
a decimal place, it should be recorded as 
096.0. This rule, of course, can not usually 
be followed if the transcription to the tabu- 
lating card is made from the original data. 


THE Key PUNCH 


By means of the electrical key punch, data 
are transcribed to the tabulating cards in the 
form of punched holes (Illustration 2). 

The keyboard consists of twelve keys cor- 
responding to the positions in a column of 
the tabulating card. As a key is depressed a 
hole is punched in the card, and the card 
advances to the next column much the same 
as a typewriter spaces as each character is 
printed (Illustration 3). Cards are automat- 
ically fed and ejected. Information that is 
commen to a number of cards can be auto- 
matically punched into each card by placing 
a punched master card in the duplicating 
rack. 


Tue ALPHABETICAL PUNCH 


The alphabetical punch contains all the 
features of the numerical key punch, and in 
addition has a typewriter keyboard for the 
purpose of punching alphabetical characters 
(Illustration 4). 
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When one of the letters on the typewriter 
keyboard is depressed, the combination of 
holes corresponding to that letter is auto- 
matically punched into the card. This 
machine is used to punch cards from which 
descriptions or names are to be printed 
(Illustration 5). 


THE VERIFIER 

After the cards are punched, a second oper- 
ator performs the same operations on the 
verifier that have been performed on the 
punch. Instead of punching holes, however, 
the verifier checks the accuracy of the holes 
already punched. Verification in such a man 
ner insures accurate transcription from the 
data sheets to the tabulating cards (Illustra- 
tion 6). 


THE COMPARING REPRODUCER 


The comparing reproducer is used to punch 
common information, numerical or alphabet- 
ical, into a group or groups of cards, eliminat- 
ing the necessity of a punch operator repeat- 
ing the same operation on each card of a 
group. It will also transfer punched data 
from one set of cards to another, either in 
the same field or in a different field. At the 
same time, or independently, the machine 
automatically checks its own punching. The 
operating speed is 100 cards per minute 
(Illustration 7). 


THE SORTING MACHINE 


After the cards are punched, the sorting 
machine is used to arrange them in sequence 
or divide them into groups according to some 
classification or combination of classifications 
in order to make the needed tabulations. 
The sorting operation is based on the posi- 
tion of the punched hole in the column which 
is being sorted. On the machine there are 
twelve receiving pockets corresponding to the 
twelve digits on the card. As a card passes 
through the machine, it is directed to the 
pocket corresponding to the punched hole in 
the column being sorted by means of an elec- 
trical circuit closed by a brush contact 
through the punched hole. For example, a 
card punched 8 in the column being sorted 
would be directed to the “8” pocket; another 
card punched 4 in the same column would be 
directed to the “4” pocket (Illustration 8). 

Sorting operations can be performed on 
only one column at a time; therefore, in order 











to arrange a group of cards in complete 
numerical sequence with respect to any field, 
it is necessary to sort each column of that 
field. A three column field would require 
running the cards through the sorter three 
times. Within any field, sorting should begin 
with the units column and progress through 
the tens, hundreds, thousands, etc. When 
sorting extends beyond a single field it should 
progress from the minor to the major fields. 
For example, if it is desired to arrange a 
group of cards in order by age within grade, 
it would be necessary to sort first on age and 
last on grade. For a group of cards punched 
like that shown in Figure 2 (Example 1), 
the sorting sequence would be column 49, 
column 48, column 47, and column 46. The 
machine operates at the rate of approxi- 
mately 400 cards per minute. 


lHe Carp COUNTER 


The card counter is an attachment to a 
sorter which performs the functions of count- 
ing (a single column at a time) the number 
of holes punched in each position in a column, 
and the number of cards. Sorting and count- 
ing can be accomplished simultaneously on 
any column. If sorting and counting are per- 
formed simultaneously for a group of cards 
on a column which contains no multiple 
punches, the counter will record the total 
number of cards, and the number of cards in 
each pocket. Even though the sorter will 
recognize only the lowest hole in a multiple 
punched column, the card counter will count 
each hole punched regardless of the number 

holes in that column (Illustration 9). 

THE TABULATOR 

Printed reports are automatically prepared 
n the tabulator from the punched cards. As 
the cards pass through the machine, the 
punched holes actuate adding, subtracting, 
selecting, and printing mechanisms by means 
of electrical contacts. The arrangement of 
the compiled and printed reports is completely 
flexible. Therefore, the form of the printed 
reports is not dependent on the arrangement 
of fields on the tabulating cards. 

The data on the printed reports can be 
automatically transferred to summary cards 
simultaneously with the printing of the re- 
ports. These summary cards can then be 
used for further analysis. The speed of the 
tabulator is approximately 150 cards per 
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minute, modified to a certain extent 
number of lines and totals to be 


(Illustration 10). 
In general there are two types 


lators, numerical and alphabetical. Each , 


perform essentially the same opera 


tl 


cept that the numerical tabulator wil! 
only numerical data, whereas the alphabet 


tabulator will print both numer 
alphabetical data. 


AUXILIARY MACHINES AND SPECIAI 


Because of their relative unimpor 


iLd 


1) 


ri9 


gaining a basic knowledge of the pri 


of tabulating machine methods of 


i 


other auxiliary machines and special! dey 


have not been presented. The prin 
operation of such machines are in 


} 


much the same as those described, ani 


are adapted to perform automatic 
of the routine statistical work in 
problems. 


ILLUSTRATIVE PROBLEMS 


To give illustrative problems f 
different types of research projects t 


ally 


researc 


r 


hat 


be adapted to machine tabulation would 
impossible. But, in order that a clearer 


derstanding may be had of the m 
which some types of problems may 
dled, a few examples of such prob! 
outlined in this section. 

Each sample problem begins with 
tration of an original individual rec 


) 


ani 
} 
I 

yer 


yY 


and a tabulating card punched fron 


record. A list of some of the possi 
lations that can be made follows, 
with illustrative reports. 


Example 1 


The objective test answer sheet 
Fig. 1 has been marked by a st 


answer to questions contained in a spec 


test booklet. Fig. 2 shows a tabula 
punched from this answer sheet. T 


half of the card, and in positions 4, 


ble 


h 


e 


r 


wr 


udent 


ting 


Lait 


he upper 
half of the first 39 columns of the card con- 
tains a punched hole record of the responses 
to the first 39 items, and the lower hall, a 
record of the responses to the last 39 items 
Responses 1, 2, 3, 4, and omit, respectively 
are in positions R, X, 0, 1, and 3 in the t 


h 


and 9 in the lower half. Hence, the responses 
to items 1 and 40 (3 and 2, respectively ) are 


punched as o and 5 in column 1; 


respon 


ses 
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items 2 and 41 (1, and 4, respectively) are 

ached as R and 7 in column 2. The R in 

40 is duplicated to all cards and 

is a check for the card punch oper- 

‘or in determining when the first 39 items 

save been completed. The card is returned 
lumn 1 before the punching of the last 
ms is begun. 


following is a table showing the card 
lds in which each of the remaining types 
ta are punched. 


Card Columns Data Recorded 
1-43 .. _.. Identification number 
$4-45 saan wore 
16-47 caune UTade 
R49 _... Age 
Sex (1 = boy; 2 = girl) 
Some of the possible tabulations are: 


item analysis; 2—frequency distribution 
res; 3—sums of scores and numbers of 
oupils; 4—sums of squared scores; 5—cor- 
ition of odds and evens for reliability 
ring to be done with card stencils). Each 
the above tabulations can be made in total 
n groups by achievement, grade, sex, age. 
r responses to items or groups of items. Such 
bulations contain the basic data needed fo: 
mputing the desired statistical measures. 
Fig. 3 is an analysis of six items of the test 
wer sheet shown in Fig. 1, by achievement 
ups for 300 pupils. The upper, middle, and 
wer thirds are determined by grouping in 
rdance with the total score on the test. 
In this case, approximately one third of the 
pupils scored above 33, one third between 27 
| 33, and one third below 27 (see Fig. 6). 
rder to simplify the computations the 
groups were adjusted at each extremity so 
t there were 100 cases in each group. This 
inalysis shows that a total of 128 pupils in- 
cated response No. 1 as the answer to item 
No. 1, 73 of whom are in the upper third ol 
the distribution of scores, 40 in the middle 
third, and 15 in the lower third. Also, 126 
pupils indicated response No. 4 as the answer 
to the same item, 25 of whom are in the 
ipper third, 45 in the middle third and 56 in 
the lower third. Likewise, a total of 105 
pupils indicated response No. 2 as the answer 
item 41, 29 of whom are in the upper 
third, 32 in the middle third, and 44 in the 
lower third. Item 1 and item 4o are on the 
same line because they are both punched in 
olumn 1, and are counted simultaneously. 





to 


w 
~ 


Fig. 4 is an analysis of the same six items 
by grade. Of the 128 pupils indicating re- 
sponse No. 1 for item No. 1, 14 are in the 
eleventh grade, and 114 are in the twelfth 
grade. In terms of percentages, 23.0% of the 
eleventh graders, and 96.5% of the twelfth 
graders made this response. 

Fig. 5 is a cross-analysis between item 1 
and item 2. Of the 126 pupils selecting re- 
sponse No. 4 to item No. 1, 118 selected re- 
sponse No. 1 to item No. 2, one selected 
response No. 2, 3 selected response No. 3, 
2 selected response No. 4, and 2 omitted the 
item. Likewise, of the 278 that selected re- 
sponse No. 1 to item No. 2, 118 selected 
response No. 1 to item No. 1, 12 selected 
response No. 2, 8 selected response No. 3, 
12 selected response No. 4, and 22 omitted 
the item. 

The first column in Fig. 6 is an indication 
of the test scores in ascending order. The 
other six columns are, as labelled, the fre- 
quency and cumulative frequency for the 
boys, girls, and total group. 

Figs. 7 and 8 are sums of scores and num- 
bers of cases for each response to item 1 and 
for age respectively. The averages have been 
computed with a calculating machine. 

Fig. 9 contains the cumulative sums of 
scores by sex for each of the separate digits 
in the units and the tens positions of the score. 
The last number (5079 for boys) in the 
cumulative column is the sum of all the 
scores. The sum of the squared scores (=X*) 
is obtained by adding the cumulative sums 
for each of the digits, except zero, in the units 
position and ten times the cumulative sums 
of each of the digits, except zero, in the tens 
position. This and the remaining figures are 
obtained on a calculating machine and repre- 
sent the computations for the mean and stand 





deviation from the _ raw-score formulas: 
— ‘SX? — M?, _ 
M y 5D { er By combining 


sums of scores and sums of squared scores 
for boys and girls, a mean and standard devi- 
ation can be computed for the total group. 
Fig. 10 is similar to Fig. 9, except that each 
score has been divided into two parts—the 
score on odd items and the score on even 
items—and cumulative sums obtained for 
each. In addition, the cumulative sums have 
been obtained for the odd-item scores (X) 








and each digit in the units and tens positions 
of the even-item scores (Y); so that now we 
have, by adding the column labelled “X”’ for 
the tabulation indicating digits for Y, the 
sum of the cross-products (XY) between X 
and Y. Using a calculating machine, the 
standard deviations are computed as in Fig. 
9, and the Pearson product-moment coeffi- 
cient of correlation is computed from the 
raw-score formula: 
=X 
N 
SD, .SD 


M.M 


Example 2: 

Fig. 11 shows two pages from a question- 
naire that is a good example of one which is 
self-coding. Parentheses indicate the codes 
and the card column numbers for each item 
of information. Fig. 12 illustrates the tabu- 
lating card punched from these two pages. 

Any type of analysis or cross-analysis of 
the information contained in the question- 
naire can be obtained from the cards. A few 
sample illustrations are shown in Figs. 13, 14, 
15. 

Fig. 13 is a distribution by age (column 1), 
and is interpreted as follows: 5 persons did 
not answer the item of age, one is from 18- 
19 years of age, 13 are from 20-21 years of 
age, 173 are from 20-24 years of age, etc. 

Fig. 14 is a distribution of items 3, 4, and 
5 (column 3), and is interpreted as follows: 
Of the 989 cases, 509 are men, and 480 are 
women; 473 are single, 469 are married, 16 
are widows or widowers, and 12 are divorced. 
The early environment of 214 cases was the 
farm, of 157 cases the small town, of 238 
cases the cities from 2000—15,000 population, 
and of 360 cases the cities over 15,000 
population. 
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Fig. 15 is a cross-analysis of age (colym: 
1) versus present population (column 7), (9; 
the 173 cases from 22-24 years of age, one 
person is teaching in a city with a populatioy 
of from 1ooo—1499, one is teaching in a city 
with a population of from 1500-1999, 4 are 
teaching in cities with populations of frop 
2000-2999, etc. 

Example 3: 

Fig. 16 is a sample of a data sheet 
which has been posted a record of twely 
measures for each individual case. Fig. 17 
a tabulating card punched from case No 
Note that numbers having fewer digits tha 
have been allowed for in the tabulating card 
field are preceded by zero. 

From these cards can be tabulated various 
distributions, sums, and progressive totals by 
digits for computing sums of squares ani 
sums of cross-products for use in the formu 
for the coefficient of correlation. Such a tab- 
ulation of progressive totals together with th: 
computations for the correlation coefficient 
are shown in Fig. 18 for a sample of toc 
cases. 

The column to the extreme right of each 
series of tabulations is used for the sums of 
squares to obtain the standard deviation oi 
each variable; and all other columns are used 
for the cross-products between the column 
that is being added and the column to the 
extreme right. This tabulation follows the 
same principle as that in Fig. 9, except that 
columns for 11 cross-products are obtained 
simultaneously. Cross-products may be ob- 
tained between one variable and as many as 
20 other variables in one set of cumulative 
totals. All computations have been performed 
on a calculating machine. 
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Illustration 1. The Tabulating Card 





Illustration 2. International Duplicating Key Punch. 
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Illustration 3. Tabulating Card Punched on a Key Punch. 
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stration 4. International Alphabetic Duplicating Key Puncl 
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Illustration 5. Tabulating Card Punched on an Alphabetical Key Punch 





Illustration 6. International Motor Drive Verifier. 
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Illustration 7. International Comparing Reproducing Punch. 











Illustration 8. International Horizontal Sorter. 





vorch, 1941 | TABULATING MACHINE METHODS 263 




















e 405. 


Illustration 10. International Alphabetic Accounting Machine Typ 
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Name .... Zo nes 
Last name 

Grade 4/ 

Name of teacher Mss ‘ 

School 

City AnyTou fal 


ANSWER SHEET: AMERICAN GOVERNMENT 


Age:. IS. yrs 


Adams... 
Pu lic. High Schoo/. 


274 


. Score 7 


Check: Boy. &. Girl 





Sample 0 C] C7 r7] 








1IOOKO 
26000) 
30600 
4MOOU 
s5MOO0 
6OOO0N 
7OMO0 
sOOWO 
2x00 
loOUKU 
nxgOO 
IZ2OWOU 
13OWOO 
40008 
I565ROOO 
16 OMOO 
I7OOUON 
1wOOWO 
1I90WOO 
200800 
20W00 
228000 
233R000) 
24R000 


23 OHOKIC) 
26 JU MU 
27 UU 
28 000%) 
2000 
30 0MO0 
s100WO 
32KHO00) 
33 0OWOO 
34K O00 
35 (WOO 
36000) 
s7O WOU 
38 DUO 
3s9M000 
40 0NOU 
410008 
42008 
8s 00K 
“0800 
450 W000 
40800) 
470 R00 
48000 
a0 R00 


50M O00 
s1C)WOO 
S20 XU 
s330NO00 
5400MO0 
sOOO0K 
560 XO 
57000 
s800UN 
ssOOUUW 
ce OKO 
618000 
62K 000 
63 080 
+4ORO0) 
6 O00 
6 OMOD 
670 M00 
680KO0 
eQOOWO 
7OWOO 
72WO00 
7200K0 
7320008 
740008 


73MOO0 
7o>OO00W 
70800 
78MO00 











Figure 1. 
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i) 
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jin ote mt us 
prt wom pane 
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Coeececceovecdceecsoogocoececy 
PERRO ROR ERODE ERE EPR EOERER EES | 
222222222222222222222222222224 
espera fe 0 ede peep 






22222222222222222222222222222222222222 

9232922299933923323333393333339333333333 
OMTTTT EEL) GS RRREEECEEE) | COCeeerr! Cie oe 
QpassSMORESMOMSHES Sos SHSSSMMMERSESssssse 
(isPECSCSSESCNSCSBSERC SOBs eco checpessss 


nT PEERRREELEAD) b) | | PRRRERRREROEED | Ob 





YVTETITTTETETETETETTTrTerirty 





SSSISSSSSSSSSSSSSSSSSSSSSS55S§ 


$6 rhe ppt phalpah tad eint 





17727277777 17 padaeaTIIIIIADE 





WHTTI TIT L Le QOGRRELHRESESSRRRRSSReREteeTER 
eqpgsgesgsgegageseggeosesagessogcgges O99SSISTGSSTISSSNSSoSasegseN 
sh span HU BEVPRBANY RAAB At x=BERYS Se aoe tae yt a6. 5 a” 
Figure 2. 
Response Response 
| 2 3 4 Omit Item 1 2 3 4 Omit Total 
Group 
Upper 73 1 25 1 40 71 2 14 9 4 100 
Middle 40 5 4 45 6 56 7 22 7 8 100 
Lower 15 8 4 56 17 2 7 24 13 14 100 
Total 128 13 9 126 24 169 16 60 29 26 300 
2 Upper 95 3 l 1 41 3 29 10 55 3 100 
Middle 90 3 4 2 1 5 32 13 41 9 100 
Lower 93 ‘ a 8 44 14 27 7 100 
Total 278 6 10 3 3 16 105 37 =1128 19 300 
Upper 58 15 13 3 11 42 54 2 14 25 5 100 
Middle 32 11 11 9 37 32 3 21 32 12 100 
Lower 17 14 6 13 50 18 7 21 36 18 100 
Total 107 40 30 25 98 104 12 56 93 35 300 
Figure 3. 
Response Response 
Item 1 2 3 4 Omit Item 1 2 3 4 Omit Total 
Grade 
l 11 14 Meudon 31 12 40 28 7 13 6 7 61 
12 114 9 9 95 12 141 9 47 23 19 239 
Total 128 13 9 12 24 169 16 60 29 26 300 
2 11 57 1 rats 2 1 41 2 27 10 18 4 61 
12 221 5 10 1 2 14 78 27 105 15 239 
Total 278 6 10 3 3 16 105 37 86123 19 300 
3 11 22 7 5 2 25 42 17 2 8 20 14 61 
12 85 33 25 23 73 87 10 48 73 21 239 
Total 107 40 30 25 98 104 12 56 93 35 300 


Figure 4. 





JOURNAL OF EXPERIMENTAL EDUCATION [Vol.o.% 


Hf 
ITEM 2 
Response l 2 3 4 Omit Total 
l 118 | 5 l 128 
2 12 l 13 
3 8 l 7 9 
4 118 l 3 2 2 126 
Omit 22 l l 24 
Total 278 6 10 3 3 300 
Figure 5. 
Responses Sum of No. of 
Boys Girls Total on Item 1 Scores Cases Average 
Score F G. ie F oe A F C. F. 1 4746 128 37.1 
12 1 1 1 1 2 345 13 96 5 
15 1 1 1 1 2 3 259 9 28.8 
17 1 2 3 2 4 4- 3610 126 23.7 
19 3 4 3 6 6 10 Omit 610 24 25.4 
20 3 7 3 9 6 16 ms - 
21 1 8 4 12 5. 21 Figure 7. 
22 4 12 9 22 13 34 
23 3 15 10 32 13 47 
24 5 20 5 37 10 57 
25 7 27 9 46 16 73 
26 8 35 12 8 20 93 
27 7 42 5 63 12 105 
28 9 51 10 73 19 124 
29 3 54 7 80 10 134 
30 6 60 9 89 15 149 Sum of No. of 
31 . 68 . 97 16 165 Age Scores Cases Average 
32 12 80 8 105 20 185 xx 697 23 30.3 
33 7 87 l 106 8 193 14 23 1 23.0 
34 6 93 7 113 13. 206 15 267 9 29 7 
35 8 101 4 117 12 218 16 2021 62 32 6 
36 3 104 3 120 6 224 17 4351 134 32.5 
37 2 106 5 25 7 231 18 1749 56 31.2 
38 10 116 4 129 14 245 19 387 13 99 & 
39 4 120 | 130 5 250 20 75 2 37.5 
10 3 123 2 132 5 255 
41 4 27 4 136 8 263 Figure 8. 
42 27 3 139 3 266 
43 l 128 l 140 2 268 
i4 3 131 l 141 4 272 
45 6 137 2 143 8 280 
46 2 139 143 2 282 
47 l 140 143 1 283 
48 l 141 143 l 284 
50 l 142 l 144 2 286 
51 l 143 2 146 3 289 
52 l 144 l 147 2 291 
54 l a” «savxca ee 1 292 
55 l 146 nae ae 1 293 
56 1 147 1 148 2 295 
59 l a, Sarasa 148 1 296 
61 l es -. 1 297 
65 ee 1 149 1 298 
66 l 150 |. .. 149 l 299 
73 l Bee ewes 149 l 300 


Figure 6. 
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Figure 9. 
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Applies to all. Indicate with @ cross (x) the age 3. Applies to all. Indicate with a cross (x your sex 
group below which best represents your age in years ' 
. } 
to your nearest birthday (0) pd —_s 
q) female 
(R) 18 to 19 years 
cts... 20 to 21 years 4. Applics to all. Indicate with a cross (x) your socal 
status : 
(0) 22 to 24 years 
(2) ~........single 
(3) eum 25 to 27 years 
(3) - x ~married 
(2) 4% 28 to 30 years 
(4) ....... widow or widower 
(3) 31 to 33 years 
(5) .........divorced 
(4) 34 to 36 years 
. - 5. Applies to all. Indicate with a cross (x) your ear! 
(5) s7 to 40 years environment: , , 
(6). 41 to 45 years (6). x. farm 
(7) 46 to 50 years (7) .-----$mal] town (under 2000) 
(8) 51 to 60 years (8) —......city (2000 to 15000 pop.) 
(1)... (9) .........-61 years and over (3) _--..(9) —.......city (over 15000 pop.) 


6. Applies to all. Indicate with a cross (x) your re 
ligious preferenc« 
2. Applies to all. Indicate with a cross (x) the age 
group below which best represents your age at the x Ror thol 
time you entered the teaching profession: (X)——--Roman Catholic 
(0) —.....-.Presbyterian 


(1) A Methodist 


(R )........-.15 to 17 years 
(2) ..........-.Baptist 


(X) 18 to 19 years 
(3) .....<...Lutheran 


(0). 20 to 21 years 
— —— Congreg*®tional 


(1) XK 22 to 24 years 
(5) —-.....-Episcopalian 





(2) ..-.—--25 to 27 years 

(6) .... Other Protestant sects 
(3) ..........28 to 30 years 

(7) .no---000--J CWish 
€8) ccnnces 31 to 33 years | 
(5) —........34 to 36 years 7. Applies to all. If you have no religious preference, 

put a cross (x) in the space below: 
]) ccceermecees 37 to 40 
“7 ‘ — (8) ~......n0 preference 


(7) ~......41 to 45 years 
8. Applies to all. If you are a member of the sect indl- 


cated above, put a cross (x) in the blank below: 


(9) » church member 


(8) ........46 to 50 years 





(2)... (9) —.....51 years and over (4) 











Figure 11. (lst page) 
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9. Applies to all. In the list below, indicate with a il. 
cross (x) your father’s oceupation while living: 
Note: Check but one item. 


(B)...AX..tarmer, agriculturist 
(X).........agent, office worker, salesman, realtor 
12. 


(0) ........-banker, financial agent 


(3) carsnseseses contractor, building trades: carpenter, 


bricklayer, plumber, ete. 
(2) —...-..government and public service 


(3) —.......merchant, dealer, proprietor 





13. 
( ) professions: doctor, lawyer, dentist, 
teacher, preacher, etc. 
(5) .......railroad employee 
(6) ..........mining (laborer) 
(7) trades other than building, (skilled me 14. 
ch » painting, decorator, ete.) 
(8) ...manufacturer, public utilities 
(5) (9) ...........unskilled laborer 
10. Applies to all. In the list below, indicate with a 
cross (x) the type of teaching certificate you now 
have registered with your county superintendent: 
(If you hold more than one certificate, indicate only 
the highest.) 
(R/0)....%....First Grade State 
(R/1)...........Second Grade State 
(R/2)............Third Grade State 
(7)... 
(R/3)...........Fourth Grade State 
7) Rural] School Teacher’s State 
(B/5)...........Special State (Kindergarten) 
15. 
) | Special State (Primary) 
(B/7) 2-20-0002 Other Special State 
(X/0)...........First Grade County Uniform 


(X/1).........Second Grade County Uniform 


..Third Grade County Uniform 





(X/2).. 
(X/2).........-Special County 

(X/4)............Provisional County 
(X/5)..... 
(X/6).........Life Diploma 


High School Normal Training 


The above list should contain only one check mark. 





Applies to all. If the certificate which you indicated 
in the preceding list was granted you by examination, 
place a cross (x) below 


(8) _X certificate granted by examination 


Applies only to holders of Wfe certificates. If the 
certificate you have checked in the preceding list is 
validated for life, place a cross (x) below: 


(6 ence (9) ccna life certificate 


Applies only to holders of special certificates. If you 
have indicated a special certificate, write below the 
name of the special subject you are certified to teach: 


Applies to all except rural teachers. If you are not 
teaching in a rural school, place a cross (x) before 
the population group to which the city in which you 
are now teaching belongs: 


(R)........below 590 
+) 500-999 

(0) .........--1000-1499 
isis 1500-1999 
(2) .. ~ ...2000-2999 

(3) ---.----0---3000-3999 
(4) ~......4000-4999 
(5) .....=-..5000-9999 
(6) .........10000-14999 
(7) --.-----15000-24999 
(8) - 25000-49999 

sem (9) ..-.0-----50000 and over 


Applies to all who have taught in more than one com- 
munity. If you have taught in a town or city other 
than the one in which you are now teaching, indicate 
with a cross (x) the population group to which the 
city belongs in which you taught just before coming 
to your present position: 


(> — below 500 
(X).. ..500-999 

...1000-1499 
1500-1999 
2000-2999 



















.-4000-4999 
.... 5000-9999 

..1 0000-14999 
--15000-24999 
25000-49999 












Figure 11. (2nd page) 
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Figure 12. 


Unans. R X 0 1 2 3 4 5 6 7 S 9 Tota 
Age 
Col. 1 5 l 13 173 182 181 131 105 75 37 31 39 16 989 
Figure 13. 
0 1 2 3 4 5 6 7 8 9 Tota 
Col. 3 509 480 473 469 16 12 214 157 238 360 «989 
Figure 14. 
Present Enrollment (Col. 7) 
Unans. R X 1 2 3 4 5 6 7 8 9 Total 
Unans. - 1 1 a 1 5 
R ane SE ae oe 3 : . l 
X l 1 . _ 6 3 eer 1 13 
0 1 | l 4 2 7 54 36 10 21 360— «173 
l 2 : 2 7 46 35 13 21 56 =: 182 
2 1 l 2 4 3 36 36 15 26 57s 181 
3 2 l 1 5 6 24 22 12 17 41 181 
4 1 1 1 5 18 20 6 20 33 105 
5 eee er se 2 l 1 13 11 2 18 26 75 
deed ag aca eee ore ea ae ee ace ae a a eee 3 2 6 1 5 20 7 
Ee Cae IE PN eee nes 3 8 1 8 11 31 
ee ; :  asecee 3 4 4 2 4 21 39 
9 ee Tee ok ees 2 1 3 5 16 
Total 8 2 oe ae 2 10 17 36 = 211 185 64 145 307 989 
Age (Col. 1) 
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<e / e 3 4 s é 7 8 G9 so Wf /2 
1 78 72 90 88 090 87 82 74 80 46 083 60 
2 90 88 78 78 O77 50 72 48 66 46 087 60 
; 84 82 78 80 080 57 72 60 60 60 073 67 
4 60 74 74 86 O87 9O 76 62 90 58 090 83 
5 82 74 84 80 090 80 88 ‘4 O62 42 097 93 
5 50 50 54 44 O8 3 8 3 60 66 72 64 087 7, 
7 74 B84 86 72 090 S83 74 46 64 66 097 > s 
o 78 80 86 90 ey F 6 O 84 3 Oo 40 5 8 100 67 
9 82 64 94 86 O9 3 70 78 3 4 BS 66 O83 80 
19 70 60 80 64 083 67 88 54 790 16 O93 £h 
74 88 80 90 86 090 93 90 66 8 14 087 93 
12 74 80 90 72 093 73 78 58 80 62 083 67 
13 84 88 82 84 100 77 72 50 72 62 083 93 
14 64 66 86 86 090 90 74 738 TO FC O67 90 
15 56 62 72 78 O87 67 70 66 68 74 097 73 
16 84 68 92 84 O77 TT 80 84 72 48 093 a % 
17 82 9A2 8S Bea O80 67 86 68 64 54 083 67 
18 90 82 84 78 067 80 78 78 66 70 083 87 
19 74 7O 82 62 090 60 79 76 7 46 087 50 
20 88 88 90 74 087 73 90 72 68 60 077 80 
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Figure 17. 





JOURNAL OF EXPERIMENTAL EDUCATION [Vol. 9, No. ; 


to 
~! 
N 


























i ) (2) 2) ) (I$) ©} OZ, @ yw Ce 
ih 2778 2636 3140 2956 $163 42717 2908 2640 28064 2302 
2778 2616 3140 2956 3163 2717 29084 2640 26a 2302 
277868 261° $340 2956 $165 2717 2908 2640 2804 2302 
2778 2616 3140 2956 $1653 2717 2908 2640 2804 2302 
4940 4600 S5o¢4 sive 5586 4800 5076 4724 4970 4102 
4940 4600 5594 Siv64 55866 400 so7z6é 4724 4970 4102 
494 4600 5s5¥4 S5is4 §586 4#00 5076 4724 4970 4102 ‘ 
 . ere ee res s3i4 70084 7545 TOTS 7390 Buss Ty 
| |9 868 832 1004 976 1045 e886 926 930 900 768 
“ 2718 2566 $108 2942 3130 2623 2790 2688 27 74 2222 
| {7 39640 36386 4400 415a 4527 3856 40286 3752 3930 3144 
‘ | Io} | S354 §008 6056 $655 6140 5294 5528 $190 5454 4424 
1 {5 6200 54838 7136 6567 7250 6200 6504 6064 6460 5188 
“ \4 6558 6178 7552 6955 7653 6517 6668 6408 4624 $516 
=z 113) 6974 6560 8030 7403 8073 6920 7324 6824 7206 5486 
la 7034 6612 80970 7475 6150 6977 7398 686888 727A $940 
| 7148 6732 82246 7599 8314 7094 7548 7014, 7390 6054 
‘aa ae 6 2 o 26 7S99 e314 T7094 7546 7014 TITI0 6054 
rr TIas 6732 Boas #7599 #B314 #7094 £7545 7016 7390 O05< 





——_— ++ ————_$——————————— _ 


7: ae 
| 

Zetex | | | APH OTA, Voted S45 em Lathes S12alo  Mhagst Sibteo #tasa Soret,  ¥/Ag34 56/970 g7245 
Th Pity C94 0.7d. thts? Shh bh SAtkshle S7ada/O #99939 Sthdsdd €4A9/R S¢7M Bb #/AG AY F413,76  ¥9724! 












































—A_ tt ~  - 42.45 
_30 | ————— cinnsemmnmenebnaiatanian Lh Lids 
| 
3On" Sy | Jig. /6 204,06 203,94 2010 20044  2/7.98 (70.71 210? SAE 166,57 Ax 
ae imma 3030 .d8/6 2709 +3299 -dgeé #5 /6  -ASS/ 1 J267 244 shL3s- 
— 
Figure 18. 
l|P 1446 1744 41448 1474 21448 in2 livyve 1790 19964 15664 224A 
| | } 1Aa4e i744 fine 197 2188 i”“7s5 igyve i7v0 1994 1564 4 
1] 1a4ae 1744 144 1974 2148 145 19964 1790 1994 4366 ‘ 
aan 1846 1744 low 1972 2184 Lees 1994 1790 19964 1564 ‘ 
| 3 4402 422¢ 5¥10 4766 5144 4406 4748 4372 4610 3794 ; 
j | 440 4. ¢ 210 4746 5144 4406 4748 4372 4610 3794 
puff tn 4 62 4394 300 446% S217 4485 48924 444 4492 3870 S091 
Tian 6ST 8 , 73599 31 4 7094 7548 7014 390 e054 ; 
4 49a 45 4 0 736 3061 2571 656 2498 2472 >0 74 TP«s 
al | 467s2 43 3° sg 7 5447 4667 4u42 45664 4794 34H 42 
7 754 $1 661 6105 6676 5735 AO0VU6 §554 6944 4762 ‘ 
6 6196 ¢ 7229 62404 6468 60354 t »0O 5220 
rae o ’ 7843 6750 O74 650 694 712 é 
4 6a74 6 4 tH 7? ) 7vAa 6 650 72%34 € 8 7124 432 
’ 6934 6 1014 Sn7 4063 6¥17 7316 67°90 716A 59 ¢ 4 
498 é a 441 4150 6ve67 7384 64 4 7236 5946 
__ 6988 65 ? sz 451 — 150 64 7 7 38 464% 54 72 56 99 46 ? 
| 1448 ¢ 5 49 4314 7094 7548 7014 7390 6054 } 
| 160 1 ‘ 7 « lok 164 12 164 4 ’ 4 " a 
wii. Tt te rot TOG st 7 TESS TS 377A? - aed ~ a, 
—— tt - it pemevenaaeamaty ‘ 
DQvrorkayl | | | Sesto Stra 659692 k/0gsa 666702 Steere 60439 Stosew So 324g vbeisd $574 
! 
2 |) | sects Samra  459sga 6/0962 bhener Seger £02536 Stohet 59368 Whose bSep.ty 
74. 
REE i — hl, 
i} 
3D _ $4047 
— ++ - - ee 
A er 7) ee ee A 2) a? a?) eee oe eee a) A) 


— = Lit tee —__.akhit sve Vote Je PBS ct Atl (S72 ef EEE 





























TABULATING MACHINE METHODS 


















































~J 


we 



































. 
sea? D® 7 4 > q 
: ~ et ade } : 
“* “a€@n ood » °® ccage ceovnnoecene! FY JTS | 
77 one Amoocorerr 4d O ~Me WM Covcedeeo Bs 3 tm 
‘ s ope owe! & Ws 
> ¥ ake mereees 8 GN s 
. } ’ + 
5 gootonoovoe a 1 
Ane MMR eeee x 3 » so § i ¥ 
<o° TAAON TH OUM Cw _D2avvune gaaowreres Ss «a “ 
. ‘7 x emo eo "| ho 
1 aannee coment 4 | 
ec aworo 4dr . 
ee . 3 s 
ee > . xs ‘ , 
oomrne » + oennenoage <o v s + 
. - amend a @ 4 = 
. - : s & . 
~ Prow om wo ~~ % 
a Ff espe - Is 1 
« Cent tatere & ~! 
eeedea Dede . ~~ & d . 
>> “ oroec,r » Da- - «> ONOBRADD bal C 
Anreec y «| “ . AaARXaD x -& "} 
ee oe ano < ¢ oa One CS & ' 
oonr "er ie tne N 8 ~ 
~~ > aan oc Sorerere » 4 ; 
aan > SICMOR HAAR : 
49 DARD , ~ 
te Ss @ 3 P 
ee na Sone ‘ Ne - oo x ee2rortooce 2 ~ | : 
ero ANSE EE SS . aa a a2 220 = ¥ a) 
-cweoe ODM Deter a + % Ss 
- ~! a Henne oe YOreogest2ao “ «© | 
et ewes oc . * : | 
wee x as e 
¢ Saal oe \: n sa | **cc rnewn a~eccececne + s =) 
7**o Ld t v77Vtoor e NY * 
i, * aaa ee WN OOOoO© oO » 4 i 
7 a a A 
— eUNNON h. ~ 
n ee te Ae a . , 
+ 
>mmic ORM Oe eo eS ; eeOoOeeOG® wooeccoeree Py % % 
. c UVOO OO OOS > * etcouneee earacowere | Y 4 1 
} OOHet nt © Oo Ge 7r<o Ee ell Ay ¥ - j 
} . | “anne enue Tr ee a i 
roat eanterrrruge = yl . 2 } | 
cae a@nnngaters = | *) | | it 4 i 
our 200 R80 ts ~ i ~ | i doe ck Ba | T T } 
| ane SEEMED ETEK! PET VOUT SS a 
j - + + +——+ ~ ~ 
= Ss J + + + + + a 
T po onen Tf } a Te | Wi se Elia 
l 44 + + +—+ ew 7) « x 
‘ « 
= a 
ie) 
A wn " ; 
= ££ a 
mae 
a 
” 
' ' 
: tometer 4 + 
Ora Owe si 4 P 
om > 7 oe . 
> Renoocood 2 _ «= 
¢ ANNO OODS : a « * _ 3 
+ *} 4 + * a | : 
| “ aan eeex ° 
aoa eceeooed 43 Sy 
> whet a Que p ? 3 w ed 
rn amon nnmey. » n ‘ ° S 
AnNOret » 2 N S 
| 4 | ¥ 
700 tree s ~ 
- ‘ ‘< . » * 
4 3 - 7 ; 
\d , © ? t | 
» -_ t . ‘ 
7 “ 
| - oy. < ¢ 
a& * 
ecoorrrs| 2 » : 3 
e*« avenecece ys : N 
44 ° ©2° vu © ° ° 
< “rh + * ~ < 2 . 
| : os 4 
n« Atotttre ~ a . 
we > Woovoo°e = @ a 
e mux ~ " ~ 
_ ane . > ? ” . ° 
| 
. 
arle evneseesde| Bs 
@ Oe tt etele | “ ‘ ~ 
¢« wANON mmmin > : 
ae = " 1 
| « | 
DADA UMMA AIR ~ 
. . . 
> HMRC ro a | 
ma "ox 6 > : | . + 
nox cee e * | , 
| | 
| | ? < 
_ 9 mene cool yy 
c ows Ps . 
a" 4 ©car ~ = . 
- o« meu nf | . evr? 
“noe wet © ome > | > : 
| | bad > 
| J 4 | ‘ i 
oe Oore wwe a ° ye 2 
aoe wonammmnin| Fs = % 
* me meen eens : y~ 4 ° 
ew aM oo OOW!K * | » ce«wee - ete 
| j a nee BS 
| | } + + 
| | | 
7 T2S Ww ONO ® O8E\c _ 
COnNe ¢ OOO Se eeele . t ” - + 
SN OM Ft tetris ~ e ¢ = a ; 
724) mOm eels | » M } ‘ 
| | i 3 4 ‘ ; 
| | = 
}__ — } 4 * a 
| oe | | | ] a| ¥ 
j ou } ; } ; } 4 a x 
L } } } } j } } < 
= _ 
* a an 
w ai “ 
- o . & 
- a 
Ww “ 












































—) . , . 
. alae 1 +> 7 2 ; = 
— : ~ ] aatts 
- . » | ssa * ee coownee so 4 } | j 
— ameeNNCdm w ‘ 3% ow 6 car | , § . | 
s « 1 . " a . 
eeeewoann 3 ¥ % s ne nmin we x » . cove ocoage de tpeedntedindte = 
208 yuu e092 ~— ‘| | AQQMOrenuMes CtManeee ele Dc ey 
ae eee ® P| > 3 1 od ~ w MONMAORCOMMA NH OMNodd « oe wv ~ 
MACHO At OOK c le e COT ONNOC Ge a 6 y MS AAs eo ole Toren eee ey i 
| 4 coorrrne hs q ™ 
> | » | UNH eC de ~ a + 4 a - =— SS S 
= co anew *42ox ay x 3 4 ~ ot ram neeOie a x 1 a a a —_ 
~ NaTernwuie Howtnnee de |+ ow " YW | | ante Dersas a“ -~ 
ed COPA WE NH 2 nO aadea a bs + ot 4 } } —_— 
~ Lalani ins | 7 ¢ y > ou) a a +--+ + + ‘ | | 
e } | | | 
~ } — + } | | | 
- 4 
= e*eo? | | | | | | | » ~ | * = = $ 
S pee PERE EEE Gers’ nb 
. — a = £ S z i || 
~ ~ a ~ A é 
~ | w r bl | 
He Sz FE] al 4 2 wy 
mx) sw “ 
r 4 
) 
aay 
— 
“ 
c 
™ 
aa 
xy 
~ 
eS 
~~ 
< speneygy qoesceees] 94 yy | 
onn"enda « en ®Ooo a >» ; 
- Quanecado ansance rsqaqg4a | 
— nnendedn yeooreer iw & * } | NONHTIye aHoversTeeT oe | a 
La | ’ | Het ASOD BOCHaeatwtades Ce | y RY 
= Se | iy ooo escoe me bh hee ab hs Tiss 4 | POBOUHA WA COON HNOAD HO 4 zy 
mL OUNNHEHT Moronteres Cia. | ; | & | Se | iad . > 1 CODDONMNAMA ONAaroar as 2 x 
eS PHO et tee S ws } iww } | ”~ ~ ~~ CLSCONMNOEGM BAN KT HHH AO ¢ & 2 
e MoCornnr OHO “er " ‘ » % aamnee ft NOt eer e He . 
x nae take > a > | 1 DOO OM MND Lae eee ° 3 : > ws 70 © | 4 1 
a ot | | ~~ CAP CHEE EEA cD a ° . 
~ | } | ee - rw ”~ 2 . " a ¥ +> 
s 3 a @ ” o> Galen b ~ DOCONSOHSe CONN? Cows © | 
eu Nane eo porn >R'D a4 | 3 ; e . 4 “ oeor oe en One © ame yn | z ” 
. , Arxva> “ . | ® We & o e te 08 ] 
~ va es hate | - q s. 3 | a ¥ aw » & now onto meee | _ 3 | v 
= . 3 * basin seeceeoose a re - * . “- ee 76 ©ooo a “ | 
Am | ve %y Be < z oe < | 
hia. i; | q 6 te 80 OG 80 CER ERE * 3 y Sis | | | gl = 
= da a PETeHnun CoCocereed ne 4 NS ‘ YOON TOD cowenena ne & J _ 
~~ eeocvoa eee a | ~ UN NY > WOeeeeer eee x | RCP AMMA SORA CEnnA 4 a q 
— TH MH On> 7OC.¢ | = is a S&S AA eee COO FAN Hore Me 5 4 
. oe core oe | } yy 7 ~— F 3% s @ “HH ANNAN MN CCCCe ao a & 7 
_ ann“ ane sere a «| | “ wevervewn IN oN dpodien ’ ” . 4 ] A 
| Nv ww wre onvoar “wwwr or Mi ey t . ~ 
~ naAne et ’ e 4 
~—< | " fee Lp fede teh he * Wy © «cece code ov Ceoneoge ss s 
@ttrtcee 70% 3 5 as oer eee i< Rt indies - He eOR NAN ON KMne eee sé “ 
eooonee Ahad ne + 3 bh q OOM ROO ANA OFf6 6 Onn oe Ss w ™" y 
Wanses “anee < 7 y - ie SRNNooge FGoecocooeoos o q z + a : ve 24 "on eeone Le a “4 SS 3 
| Adtawede OF eorerene me fal ¢N } 
wow x G1 Vn COOTT Ge a >| a ss NS —s - ——— + 4 + ; ‘ ; 
fcoccr+cc a<<es - J q 3 « ww 7+. *o< oétecvor rt a | ‘ ‘ a | Ewuee | . ! > I T 
CF er neds aenee «+ , } I aS Ee | ae ee | 
CH HHOONH COnne oe , § be —_—— = 4 | ; } + ; ~ ol 
nnnmnete Ne " 1 . + TT: + +4 , ; , 4 3 a 
= =< + ; _ + . 
| ——f : t+ ++ Je rl ol ly 
; 4 = =o 1... 4 ; 3| . 
» Pim i , , ; ; + ; < 4 o 
; PSC oe + ; ; } ; ; 4 
I I = See Ses sor <i ie wW , 
io we s * x 
a a 
Wzrfra r “i 
+ wh “) a 
‘ 
™ s 4 
~N Ww . 








CORRELATION COEFFICIENTS AND OTHER CONSTANTS 
FOR COMBINED AND SEPARATED DISTRIBUTIONS 


Douctas E. ScaTeEs 
Duke University 


There are numerous occasions when one 
wishes to obtain correlation coefficients and 
other series constants from several populations 
combined into one, or from a breakdown of 
a general population into two or more sub- 
populations. It is somewhat arduous to make 
new tabulations of the combined or separated 
populations. There are methods by which any 
number of tabulated distributions may be 
combined without additional tabulation or un- 
due labor of calculation, and by which any 
tabulated distribution may be separated out 
from a compound table without further tabu- 
lation. The methods are equally applicable to 
calculations based on paired series which have 
not been tabulated, but for convenience of 
expression series will be spoken of as though 
tabulated. 

A method of accomplishing this result was 
published some time ago by Dunlap,’ and 
his method is called for when only means, 
standard deviations, and correlation coeffi- 
cients are available. When however one has 
access to correlation work sheets the method 
herein presented? will probably be found more 
facile—especially when more than two sub- 
groups are to be combined or separated. It is 
ilso highly flexible when certain special pur- 
poses apart from a simple combination are to 
be accomplished. 

The method herein described is simply to 
sum, across the various groups to be com- 
bined, the respective totals which are normally 
required for the calculation of the desired 
constant in any single group, and then employ 
these sums in the formula in the usual way. 
For example, in the case of correlation for a 
single group, one commonly obtains six sum- 
mations: N, =x’y’, 3x”, =x’, Sy’*, and Sy’— 
the primes indicating that an arbitrary origin 
has been used. To calculate the correlation 


_' Dunlap, Jack W. “Combinative Properties of Correlation 
Coefficients,” Journal of Experimental Education, V (March, 

7), pp. 286-88. 

*This procedure was worked out and described in the 
writer's doctoral dissertation: Selective Admission and Selec- 
tive Retention of College Students at the University of Chi- 
‘ag0, p. 107-30. Doctor’s thesis. University of Chicago. 1926. 
saS Pp 


for two or more groups combined (as though 
tabulated together on the same table), one 
needs only to combine the N’s for the two 
or more groups, then combine the =x’y’ values 
for the two or more groups, and so on through 
the six summations. It is assumed only that 
the same working origins have been used for 
the different tables, and that the class inter- 
vals are the same; if they are not, corrections 
can be applied which will be described later. 
These combined summations are then used in 
the correlation formula just as though they 
had been derived directly from the compound 
table consisting of all the subgroups tabulated 
into one—for they will be the same values 
that one would get by actually combining the 
tabulations into one. If it is a compound 
mean or standard deviation that one wishes 
instead of a correlation, he combines only the 
summations which are needed, and substitutes 
them in the appropriate formula. 


The procedure holds equally well for the 
correlation formula based on _ differences, 
s(x’ — y’)*, instead of products, =x’y’: one 
sums the appropriate totals for that formula. 
In general, any form of correlation formula 
may be employed, so long as the working 
origins and class intervals are kept uniform, 
or the differences corrected for. Formulas 
which call directly for deviations from the 
actual mean of each subgroup do not keep the 
origins constant (unless all the means happen 
to be the same) and hence call for a correc- 
tion appropriate to differing origins, as ex- 
plained later. 

The rationalization of the procedure may be 
set forth as follows: if several separate tables 
which are to be combined were actually com- 
bined by tabulating them into one, the fre- 
quencies in the subtables would in effect be 
combined cell by cell in the compound table. 
Assuming constant working origins and equal 
intervals, any one cell in the compound table 
will have the same deviation value as the cor- 
responding cell in each of the separate tables. 
That is, the functional value of a single case 
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in corresponding cells (in different tables) is 
the same cell by cell; and since all of the 
functions entering into the summations of a 
table are first degree with respect to frequency 
(the deviation values alone going to the sec- 
ond degree), the results for each cell or array 
are additive with respect to the frequency. In 
other words, a frequency of seven in a par- 
ticular cell in the compound table will con- 
tribute the same amount to Xx’y’ that separate 
frequencies of three and two and two, in the 
corresponding cell in three separate tables, 
will contribute when the Sx’y’ values for the 
three separate tables are calculated and then 
added together; and the same is true of this 
frequency of seven with respect to the other 
five summations—x”™, Sx’, etc. 

Other ways of thinking about the combin- 
ing may be simpler. In combining two tables 
one may tabulate them together on a distri- 
bution sheet, using different colored pencils 
for the two different populations. If now one 
calculates summations for the tallies of one 
of these colors, and then for the tallies of the 
second color, and adds these summations to- 
gether, he will have the same totals as though 
he had counted both colors of tallies together. 
Yet the summations for each of the colors 
separately represent what he would have if he 
took summations from separate tabulations 
on separate distribution sheets. On the other 
hand, counting both colors together is what 
one naturally does when working from the 
compound table; so that the two processes 
are equal. Another way of seeing why sum- 
mations of the functions come out the same 
by either process is to imagine that each of 
the tables to be combined is on a sheet of 
glass. The several subtables can therefore be 
laid one over the other: the frequency in any 
cell of the compound table would be the 
total which one would get by looking down 
through the several layers of glass and count- 
ing the tallies one can see in that particular 
cell in the several superposed tables. In similar 
fashion, the various marginal summations for 
a compound table would be seen to be sums 
of the corresponding figures on the several 
contributing tables. The calculation of r from 
the combinations of the several corresponding 
major summations is then exactly what one 
would do if all of the layers of glass were 
coalesced into a single table. 

The relationship can be proved algebrai- 
cally, though this would seem to add little. 
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The proof amounts simply to expressing { 
the frequency in the compound table, analytj. 
cally as the sum of the corresponding frequen. 
cies of the contributing tables. Thus, for any 
one array (i.e., the same array, with respect 
to the working origin, in all of the tables). 


fox’ = (f, + fet. - + fide’ 


where the subscripts indicate the various ta- 
bles. Therefore, for the entire correlation 
table, 


Sfx’ = S(f, + fy +... + hy)x’ 


and the same thing is true for the other five 
summations ordinarily used. The additive 
character of these summation functions® with 
respect to frequency is thus seen. 

Thus far the discussion has dealt only with 
combining. Subtracting a population out js 
simply the reverse. That is, if one has a cor- 
relation table which consists of more than one 
identifiable group, he can take out any group 
and calculate the correlation for the remain- 
ing group by subtracting the summations for 
the subgroup from the summations for the 
entire group: the remaining summations lead 
directly to the correlation for the remaining 
groups. The group which is thus taken out 
may of course itself be a compound of several 
subgroups. Successive groups may be taken 
out as desired; each group which is taken out 
as a unit however requires separate tabulation 
and calculation of its six moment summations. 
The economy of the present method rests in 
not having to retabulate one of the groups 
each time; that is, in taking groups out, one 
will seek to take out (and tabulate separately) 
the smaller group each time, obtaining the 
summations for the largest possible group by 
subtraction instead of by retabulating. Once 
the worker senses the freedom which is ac- 
corded him by virtue of the additive character 
of the moment summations he is dealing with, 
he can readily work out the best means of 
manipulating the groups. 


ILLUSTRATION WITH DaTA ON COLLEGE 
ADMISSION 


The data in Table I show how a number of 
combinations can be made economically if one 
plans his work with this in mind. The corre- 
lations represent the relationship between 


® These six major summations for a correlation table are, 
strictly, moments, N a regarded as the zero moment (i.¢., 
x is given an exponent of 0 and becomes 1) and Sxy being 
the product moment. It may be somewhat more expressive 
however to refer to them as the moment summations. 
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TABLE I 
CORRELATIONS BETWEEN FourR-YEAR HIGH SCHOOL AVERAGES AND AVERAGE GRADES 
IN FRESHMAN WORK AT THE UNIVERSITY OF CHICAGO 
' if High School r N =x’y’ Sx” Sy’? Dx s , 
North Central States 
ES eroeeee . 5622 553 5,689 10,580 9,409 352 289 
Indiana . 6210 261 3,070 4,545 5,536 65 336 
ESOS . 5019 83 774 1,457 1,695 9 93 
{ Michigan . 6565 98 1,052 1,554 1,670 26 —10 
= Wisconsin . 5197 111 790 1,576 1,467 12 — § 
f Minnesota- - - - . 5529 23 313 680 400 36 —50 
- lowa..- . 6122 194 2,139 3,607 3,728 —47 190 
All N. C. States*__- 5750 1,323 13,827 23,999 23,905 381 848 
9. Atlantic States... .-- . 3366 110 724 2,497 1,875 3 —57 
). Southern States_ - - . 4756 106 907 1,614 2,227 34 133 
11, West Central States . 4996 168 1,470 3,689 2,591 171 35 
12, Mountain and Pacific 

States aes sacs . 6156 93 917 1,579 1,419 19 —13 
) All States*____-- 5463 1,800 17,845 33,378 32,017 260 941 
14 Chicago City--- . 6120 1,707 17,063 30,081 28,979 1,719 —441 
5 Total Admissions .5618 3,507 34,908 63,459 60,996 1,979 500 


*Exclusive of high schools in Chicago. 


four-year high school averages and averages 
made in the freshman year at the University 
f Chicago, for all high school students who 
entered the University during a five-year 
oeriod.* The six summations shown for each 
row are values that one needs for the calcula- 
tion of each r; and the adding of these values 
vertically) gives the required summations 
for the correlation of a combined population. 
Separate correlation tables were initially pre- 
pared for each state. The correlation for the 
North Central states (line 8), for each of 
the groups of states (lines 9-12), for all 
states together (line 13), and for all admis- 
sions (line 15), were obtained without addi- 
tional tabulation. Means and standard devia- 
tions for these groups were obtained in the 
same fashion. Thus, working with a total 
population of 3,500 cases (which had to be 
tabulated once, in various tables), the com- 
binations shown were obtained with a saving 
of 7,100 case-tabulations over what would 
have been necessary if a new table had been 
made for each combination. 

As an illustration of separating out popula- 
tions, assume that one wished the correlation 


*These data are from the writer’s thesis previously cited. 
They have not heretofore been published, with the exception 
f the correlation for Chicago high schools which a 
in the following reference: Douglas E. Scates. “A Study of 
High-School and First-Year University Grades.” School Re- 
view 32:182-92; March 1924. The data for out-of-Chicago 
bigh schools are for admissions during the five years 1919, 
1920, 1921, 1922, and 1923; the data for the Chicago high 
schools are for admissions during the five years 1916, 1917, 
1919, 1920, and 1921. 





for all states together except Illinois. This 
value could be obtained by subtracting the 
summations for Illinois (line 1) from the 
summations for All States (line 13) and cal- 
culating the correlation from the remaining 
values—a few minutes’ work, in lieu of tabu- 
lating 1,247 cases. Or, if one had not formed 
an All States total, he would add the summa- 
tions for Illinois and for the City of Chicago 
(line 14) and subtract the total of these from 
the summations for Total Admissions (line 


15). 


APPLICATIONS AND PRACTICAL 
CONSIDERATIONS 


There are a great many ways in which one 
may subdivide and regroup correlation popu- 
lations. The procedure described herein is 
adapted to combining or separating tables on 
the basis of a third variable (usually cate- 
gorical), when the population is distributed 
on the same two variables in all of the tables. 
The division of the total group into tabula- 
tions according to categories or intervals of a 
third variable is equivalent, in principle, to 
partial correlation: but combining tables is 
not equivalent to multiple correlation. The 
latter assumes that various values are com- 
bined for each individual case before that case 
is entered into a single table; in the present 
paper we are discussing the compounding of 
entire tables, not individual cases. Subdivid- 
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ing a population on the basis of values in one 
or both of the two traits on which it is dis- 
tributed, leading to “subcorrelations,’”’ is 
analogous to the present procedure only if one 
assumes that division on a particular third 
variable leads to contiguous but  non- 
overlapping distributions on one or both of 
the two traits of the original correlation table. 
The procedure herein described has no rela- 
tion to correlations between one or more com- 
ponent elements of a trait and the composite 
of all the elements—which normally utilizes a 
single population, and does not involve sub- 
groups." 

The method will be of some help in suc- 
cessive regrouping of the data to accomplish 
either simple or joint subdivisions. That is, 
one may wish to break down a population on 
the basis of such traits as sex, age, location, 
etc. If these divisions are to be made inde- 
pendently, one will start with the original 
table for each breakdown; if they are to ac- 
complish a joint subdivision, one will continue 
from the last preceding separation. In either 
case the procedure will save at least half of 
the retabulating for any dichotomous trait, 
and will save some work in other cases. If one 
foresees such analyses, he will probably pro- 
ceed in the other direction, making separate 
tabulations for groups which are homogeneous 
with respect to sex, age, location, etc.; their 
successive combination into any number of 
compound tabulations is then simple and ex- 
peditious, involving no further tabulation. If 
one wants each correlation table fully tabu- 
lated, so that he can see the complete two-way 
distribution, then there is no saving to be 
found in the methods being discussed; but 
they will afford essential checks on one’s 
tabulating and calculations, and may be em- 
ployed solely for that purpose. 


® Dwyer, P. S. “The Use of Subcorrelations in Determining 
the Predictive Power of High-School Crades,”’ Journal of Edu- 
cational Psychology, XXVUI (December, 1937), pp. 673-80 

* Harris, J. Arthur. “‘The Correlation between a Component, 
and between the Sum of Two or More Components, and the 
Sum of the Remaining Components of a Variable,’ Journal of 
the American Statistical Association, XV (December, 1917), 
pp. 854-59. 

Ghiselli, Edwin E., and Kuznets, George, “‘Short-Cut Methods 
for Calculating Raw and Corrected Correlations Between a 
Composite Variable and Its Components,’ Journal of Educa- 
tional Psychology, XXVIII (March, 1937), pp. 237-40 

Conrad, Herbert S. ‘On the Calculation of the Correlation 
Between a Single Element of a Composite and the Remainder 
of the Composite,’ Journal of Educational Psychology, XXVI 
(November, 1935), pp. 611-15. 

Edgerton, Harold A. ‘“‘A Formula for Finding the Average 
Correlation of Any One Variable with the (n—1) Other 
Variables Without Solving Any of the Individual Correlations,” 
Journal of Educational Psychology, XXV1I (May, 1935), pp. 
373-76 
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For those who are accustomed to Utilize 
punch cards and Hollerith tabulating ma. 
chines for the calculation of coefficients oj 
correlation it may be pointed out that the 
combinations set forth herein may be obtained 
more quickly than cards can be sorted ang 
tabulated on the machines to effect combina. 
tions of populations. Likewise when a large 
population has been correlated and is to be 
broken down into two very uneven groups 
it may be found more expeditious to sort oy: 
the smaller group and obtain the correlation 
summations from that group on the machines 
getting the summations for the larger syb. 
groups by subtracting the small group sum. 
mations from those of the total group. This 
is true even though one may be interested 
only in the larger of the subgroups. 


Another situation exists when one has the 
means and standard deviations fully calcu- 
lated, for each constituent population. For the 
present method this is no advantage, as com- 
pleted constants are not utilized. The con- 
stants could be converted into summations, 
by employing N, but they would be summa- 
tions around various true means and not 
around a common origin. They would there- 
fore call for a correction (back to a common 
origin), or, if combined directly to form com- 
pound Sxy, =x’, Sy’, and N, the final formula 
would involve two differences for each table 
(between its means and the means of the 
compound table). Starting with fully calcu- 
lated constants is the situation for which Dun- 
lap’s formulas were derived, and if one begins 
with data in such form, he will be doing 
practically the same thing no matter which 
procedure he follows. 


The theoretical or structural difference be- 
tween Dunlap’s method and the present one 
is that his procedure combines summations 
after they have been corrected to the subgroup 
means and then to thé compound group 
means, whereas the present method combines 
them when taken around a common working 
origin, and corrects the summations to the 
compound group means in one step, in the 
final calculation. Both procedures yield the 
same results;’ the significant difference 1s 
purely a practical one—in what form are th 
data to start with? If one has the necessary 


tWhile Dunlap did not extend his formulas to cover sub- 
traction of groups, this result is accomplished by subtracting 
(instead of adding) a set of parameters for the population in 
question, in both the numerator and the denominator of the 
formulas he presents. 
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<ymmations, the present method obviates cor- 
-ecting them to the subgroup means and it 
corrects them to the compound group means 
in a single calculation instead of separately 
for each subpopulation. It does. not entail the 
calculation of means or standard deviations 
for either the subgroups or for the compound 
group if these are not desired in themselves, 
for the method does not make use of them. 
If one has not planned his work so that the 
working origins and intervals are equal from 
table to table, and adjustments for these in- 
equalities are necessary, the operations be- 
come more involved, and Dunlap’s formulas 
mav call for less calculation. It should be 
considered, however, that the present proce- 
dure permits readily of further combining and 
separating beyond the first operations, and 
after adjustments have once been made to 
bring working origins or class intervals in line 
these are no longer an obstacle. The formulas 
involving Nroo, or Sxy, on the other hand, 
require the calculation of new means for the 
compound group, and the consequent changing 
of two differences in means for each table, for 
every subsequent combination or separation 
of tables. 


\DJUSTING FOR DIFFERENT WORKING 
ORIGINS 


If the same working origins were not used 
in calculating the deviations on the various 
tables, the summations for each population 
can be adjusted by the following procedure. 
If the origins actually used for a particular 
table are to be moved by the amount d, or d, 
in order to make them the same as the origins 
for other tables, the summations for the table 
will have the following corrections added to 
them (the sign of d varying inversely with 
the direction of the shift). 

To 3x’y’ add:* d,3x’ + d,3y’ + Nd,d, 

* If one is using a correlation formula employing a differ- 
ence term in the numerator, as (x —y’)*, he may make 
the adjustment for a change in origin in any of three ways 
It seems most expeditious to adjust for several terms of the 
numerator jointly. Since Dx’y’ = (%) (x2 + Ly? — T(x’ — 
y’)*], assuming the same origins and intervals, the adjustment 
given for Sx’y’ may be applied directly to the three terms 
in the bracket as a unit, instead of adjusting them individually. 
It is permissible to do this for each separate table, before the 
values are combined with those of other tables, if one con- 
tinues all the way through to treat this combination as a unit 
Note that if this part of the numerator is not divided by two. 
twice the adjustment stated will be called for. The other term 
ff the numerator, usually S}x’Sy’/2N, receives its attention 
n the final calculations, after Sx’ and Sy’ for the individual 
tables have been adjusted and combined. : 

A second form of adjustment for the term >(x’ — y’)? is 
to make an arithmetic combination of the values worked out 
for the first three expressions given in the list: since 
E(x’ — y’)? = Bx’? — 2D x’y’+ Sy”, the desired adjustment 
may be obtained by adding together the adjustments calcu- 
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To 3x” add: 2d,3x’ + Nd,’ 
To Sy? add: 2d,Sy’ + Nd,’ 
To 3x’ add:** Nd, 
To Sy’ add:** Nd, 


The sign of d must be preserved: if the new 
origin is smaller than the old, d is positive; if 
the shift is to a higher point on the scale, the 
amount of the change will have a negative 
value in the above expressions; but note that 
Ndd and Nd? will always be positive. The 
value of d must be expressed in the same 
units that the summations are: if they are in 
class intervals as units, then d must be simi- 
larly expressed before substituting in the cor- 
rections. The values and signs of d, and d, 
will of course be independently determined, 
according to the amount and direction of 
shift. If only one origin needs moving, to 
make it the same as the origins used in other 
tables, then the d for the other origin in that 
correlation table is zero, and two of the above 
five adjustments become zero, and the one for 
the product summation is reduced to a single 
term. The =x’ and Sy’ which are called for in 
the first three adjustments are used just as 
they are obtained from the correlation table— 
that is, the values for =x’ and Xy’ are not 
first corrected according to the two lower cor- 
rections, as that has been done already in the 
algebra. =x’ and Sy’ adjusted by adding Nd 
to them are employed only for combining di- 
rectly with other similar summations from 
other tables. The adjustments shown above 
lated for Sx’? and Sy” and subtracting from them two times 
the adjustment for Sx’y’. A third method is to make this 
combination algebraically and calculate its value directly 
2(3x’ ry’) (d,—d),) N(d, — d,)?. If one is using a 
formula for r which contains a sum term, as 5(x’ + y’)?, 
similar plans may be employed, with appropriate changes in 


signs. 

symbolism Sd is sometimes written in lieu of 
=(x’ — y’)*. Such a d has no relation to the d used in this 
paper to represent the amount of voluntary shift in origins. 


** The worker may note that, if d is taken equal to the 
distance from the working origin to the mean (M A), the 
value of the adjustment for Yx’y’ becomes equal to the cor- 
rection usually made in the product term (namely, ©x’Sy’/N) 
to correct it to the means of the group. The same thing is 
true of the adjustment made in the square terms of the 
denominator. Such equivalence shows the relationship between 
these formulas and the usual corrections, the present ones 
being more general; the two are equal only when d has the 
value stated, whence d, = Yx’/N and d, = Zy’/N. One 
may note further that, when adjusted, the full value of Ex’y’ 
— (=x’Zy’)/N remains the same as before the adjustment— 
and that the value of the denominator remains the same. 
This is as it should be—the correction must vary properly 
to offset the effect of changes in origin on the summations 
The principal terms and the =x’ and Sy’ must all be ad- 
justed for each table however, in order for them to have the 
proper weight in the compound. 

The value, Sd, as called for by the Cureton—Dunlap hand 
and machine correlation charts, and possibly others, has no 
relation to the d used in the present paper, but stands for 
S(x’—y’), or Ex’ — Dy’. When it is called for, adjust- 
ments to it will automatically be obtained by adjustments to 


zy’. 


=x’ and 
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are essentially Charlier’s expressions for 
checking by changing origins. 

The summations may of course be adjusted 
for transferring their origin to any desired 
point, just so all the X variables in the several 
tables to be combined or separated have the 
same origin, and all the Y variables have the 
same origin. One may, if he desires, calculate 
the means for the compound table and use 
these as working origins. Little however is 
gained by so doing. 


ADJUSTING FOR DIFFERENT CLASS INTERVALS 


If the several tables which are to be com- 
bined or separated have class intervals of 
different widths for the same variable, adjust- 
ments can be made in the moment summa- 
tions by applying a ratio. The value of =x’ 
based on a class interval of ro will (theoreti- 
cally, at least) be just half as large as =x’ 
based on an interval of 5, and the value of 
Sx”? will be one-fourth as large. To make 
changes in the summations for changes in the 
class intervals, one may proceed as follows: 

If h, and hy, represent the class intervals 
used in any table, and i, and i, represent the 
intervals which it is desired to change to, 
derive the ratios h,/i, and h,/iy. For sim- 
plicity of symbolism let these ratios be repre- 
sented respectively by a and b. Then 

Multiply Xx’y’* by ab 


Multiply =x’? by a? 
Multiply Sy’ by b? 
Multiply =x’ bya 
Multiply Sy’ by b 


In other words, multiply the observed sum- 
mation by the ratio of the width of its own 
class interval to the width of the desired in- 
terval, using the same degree of the ratio as 
the deviations have. 

If one has to adjust the summations from a 
table for both class intervals and origins, it 
seems more straightforward to adjust for ori- 
gins first, expressing d in terms of the old 
intervals. If however one makes the above 
adjustment for intervals first, the resulting 
=x’ and Sy’ values will then be used in mak- 
ing the adjustments for changes in the origins 
as given earlier; also d will be expressed in 
terms of new intervals. 

* The adjustment of [(x°—y’)? may be accomplished by 
multiplying it by ab and then adding to it the following: 
a? yx + b*Sy’* — ab( Sx? + Ly”). As in the case of adjust- 
ing for a change in origins, however, it is simpler to treat 
that part of the numerator, (%4)(Sx’? + Sy’? — B(x’ — y’)?], 


as a unit; the adjustment then becomes the same as for 
=x’y’, namely, multiplication by ab. 
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ELIMINATING HETEROGENEITY: CoNnTRO! Live 
VARIATION IN MEANS OF SuBcrRoups 


The procedure described in this Paper js 
well adapted to accomplishing certain specia 
purposes. Differences in means of subgroups 
may contribute positively or negatively (a¢. 
cording to their position) to the correlation 
for the compound population—so that they 
may cause the resulting r to be higher or 
lower than any of the initial r’s. If the com. 
bined population is regarded as homogeneoys 
for one’s purpose, this effect on the correla. 
tion is accepted as a part of the normal sitya. 
tion. If the combined population is pos 
homogeneous for one’s purpose, one may wish 
to remove from the compound correlation the 
effect of differences in means of the subgroups. 
This can readily be done while carrying 
through the procedures outlined. It can be 
done in one direction only, or for both of the 
traits at the same time. 

To remove the effect of differences in means 
in one direction only (say the x means) of a 
number of subpopulations which are to be 
combined, one corrects the moment summa- 
tions containing x for each individual table, 
back to the mean of that table as origin, 
before combining the summations of the sev- 
eral tables. For example, using the traditional 
correlation formula, one would apply to each 
individual table the full correction® for the nu- 
merator, =x’Sy’/N, and also the correction for 
=x”? in the denominator, namely, (=x’)*/N. 
The summations for the several tables to be 
combined would then consist of: Sxy, 3x’, 
xy”, Sy’, and N. (Note the first two terms 
do not carry primes and are deviations from 
subgroup means.) After the adding together 
of these values for the several tables, r is 
calculated from the formula 
a =xy 

Vix? By? — (3y’)?/N 
Differences in working origins for the X vari- 
able, from one table to another, need not be 


_ © While this full correction does have the effect of correct- 
ing the numerator for differences in means in both directions 
it is nevertheless not an excessive correction for one direction 
only. This follows because the correction for one direction 
only is the same as the correction for both directions. That 
is, when deviations are taken around the true mean of one 
trait, the entire correction becomes zero, regardless of whether 
the origin for the second trait is taken at the mean or else- 
where. This is obvious from the formula: if either Sx’ or Sy’ 
becomes 0, the entire correction, Sx’Sy’/N, becomes 0. This 
condition may be rationalized by noting that shifting the 
population orthogonally—along the direction of either axis 
(as when one origin is not at the mean)—does not have any 
effect upon the correlation, and hence does not affect the 
numerator. 





r 
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-orrected for, since the x’ summations are 

erected to the mean of each table as origin. 
Differences in class interval will be adjusted 
for as previously described, except that the 
sx’ term has become zero and needs no cor- 
section. It may be noted in passing that cor- 
rections could be made to some other point 
than the subgroup mean in the event that 
there was reason for doing so—such as when 
trye subgroup means were known and the 
observed means were desired equated to them. 

The above formula has the effect of cor- 
recting Sx’? to the subgroup means and cor- 
recting Sy to the mean of the compound 
group—the former being done before combin- 
ment summations from the several tables. 
(The signs of the various summations must 
be carefully preserved in the combining.) The 
results will be the correlation one would ob- 
tain if the several tables showed identical 
X means, but differed in their Y means. This 
is not the same thing as though all of the 
tables to be combined had their working ori- 
gin in the X direction shifted to the actual X 
mean of the compound table—it is true that 
there would be no corrections called for in the 
numerator or in the =x*, as in the present 
case; but shifting a working origin has no re- 
lation to shifting actual means. Shifting the 
arbitrary origin has no effect upon the out- 
come; shifting the means so that they are 
equal produces a changed situation. If it is 
desired to equate both means of all the ta- 
bles, then all three of the usual corrections 
will be made for each table and only Sxy, 
x’, Sy’, and N will remain to be summed. 
They will yield r directly without further cor- 
rections of any kind for origin, though ad- 
justments for differences in size of class 
interval may be called for if different class 
intervals were used from table to table in 
obtaining the original subgroup summations. 

One will make assumptions of these kinds, 
and resort to these procedures, when he de- 
sires to avoid introducing inappropriate 
heterogeneity into his compound population. 
It is a matter of somewhat difficult judgment 
to determine when this situation arises. One 
is always faced with the question, If the 
groups are not homogeneous when combined, 
why combine them at all? Why not calculate 
the separate correlations and stop? The com- 
bined population may iron out erratic fluc- 
tuations in a smaller group; but it may also 
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cover up distinctive tendencies of distribution 
in the separate groups. The utilization of 
combining formulas therefore for groups 
which do not naturally belong together—es- 
pecially when corrections are made for dif- 
ferences in one or both means—must be done 
after careful consideration. For example, one 
may have separate correlation tables for boys 
and girls. Boys and girls form a natural popu- 
lation when combined, and combining the 
tables is therefore legitimate. But to equate 
the means of the two groups before combin- 
ing, by the methods just described, would 
raise a serious question. The result is a cor- 
relation that would obtain if the two sexes 
were equa! on both means: but they are pre- 
sumably not, and one therefore has the obli- 
gation of explaining why that correlation is as 
acceptable as the compound correlation based 
on observed (unequal) means. 

The method of equating one or both means 
is available for partialling out the influence 
of an additional variable when that variable 
is chiefly responsible for changes in the means 
of one or both of the traits entering into the 
correlations. Thus, time may be held con- 
stant, when it is known to be the chief factor 
influencing the means of successive groups, 
and when the concommitance of variations in 
short intervals of time is of chief concern. 
Similarly age, or grade levels, of different 
groups of pupils may be held constant—if 
there is justification for so doing in a com- 
pound table. This type of partial correlation 
is not restricted to a quantitatively expressed 
third variable, nor to a linear distribution of 
means of subgroups. The individual distribu- 
tions for specific values of the third variable 
give analytic, specific correlations; the com- 
pound correlation gives an average, similar to 
any partial correlation. It may be used where 
averages are what is wanted. 


CONTROLLING VARIATION IN DISPERSION 
WITHIN SUBGROUPS 


The standard deviations of groups enter- 
ing into a compound correlation do not raise 
or lower the resulting correlation directly, as 
means may do, but have the effect of weights. 
That is, they behave much like N does in 
this regard. They could not produce a com- 
pound correlation greater or smaller than any 
of the original correlations, as differences in 
means can; they simply weight more or less 
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the degree of correlation which is contributed 
by any one subpopulation. Being able to con- 
trol the standard deviations may be of larger 
interest through enabling one to equate dif- 
ferent scales of measurement that may have 
been used. 

Equating, or removing the effect of varia- 
tions in, the standard deviations of the dif- 
ferent subgroups may be accomplished during 
the process of combining, and is similar to 
changing the value of intervals. The standard 
deviation in one trait only, or the standard 
deviations of both traits, may be controlled. 
First, we may assume that one wishes also to 
hold the subgroup means constant, and that 
the summations for each subgroup have been 
corrected to the means of that group. Then, 
to equate the standard deviations for one 
trait only, Sxy and Xx? for each subgroup 
may be divided respectively by o, and a,’ 
for that group.? To control both standard 
deviations, Sxy is divided by the product of 
the 2 o’s, and Sx? and Sy? are divided re- 
spectively by o,* and o,’—giving VN?, or N, 
in the denominator. These divisions are made 
for each subgroup. Inasmuch as the subgroup 
standard deviations have all been made unity 
by these divisions, only two values need to 
be combined in order to obtain the compound 
correlation: N and Sxy. The correlation co- 
efficient for the compound group is given by: 
r= Sxy/N. 

It is of course possible to equalize standard 
deviations by reducing all of them to some 
other value than unity; in this case the de- 
nominators will not reduce to N. It is also 
possible that one would not wish to equalize 
them, but to modify only selected ones. This 
offers no technical difficulties. All, or selected 
groups, may be multiplied by any desired ra- 
tio—so long as the ratio is squared for multi- 
plying terms in the denominator which are 
under the radical. 

In the foregoing the summations were all 
corrected to the subgroup means, which has 
the effect of equating the means. One may 
however wish to modify intragroup dispersion 
alone. without changing the means. Practi- 
cally, the best procedure is to go through the 
foregoing steps for each group and then shift 

» Subdividing Ex? by o yields N, the population of the 
subgroup. In reality therefore the division need not be ac- 
complished except for purposes of checking; N may simply 
be substituted for Sx*. For all of the subgroups combined, 


this gives the total population, which is used in place of >x* 
in the final calculation of r. The formula becomes 


Sxy/VNZyY r Sxy/No,. 
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the summations concerned back to the com 
mon origin for all groups, by adding (for 
each subgroup) =x’Xy’/N back to the mod. 
fied Sxy, in order to create a modified Xx'y 
for the subgroup, and by adding (Sx')2/\ 
back to the modified Sx*. All six of the sum. 
mations for the various groups are then com. 
bined and r may be calculated. The reason 
for shifting the summations from the common 
origin to subgroup means, and then moving 
them back again to the common origin after 
multiplying or dividing, is that the deviations 
from an arbitrary origin contain a portion 
which must not be altered. While the Sx’ 
and Xx’ may both be changed by the same 
ratio for a single distribution, without error 
they lose their proper weight when they are 
combined into a compound of summations. 

There is one caution which should be ob. 
served. When one multiplies the moment sum. 
mations for a given correlation he is doing 
more than changing its weight among a group 
of correlations. He is also multiplying the 
random errors in the original data and prob- 
ably will not obtain the same numerator that 
he would from actual observations which had 
the same standard deviation as that to which 
he has mechanically changed his summations 
This error may, under some conditions, be 
offset by utilizing Kelley’s formula for the 
effect of range of talent on r, or Pearson's 
formula for curtailed distributions." From 
one of these formulas, the appropriate sub- 
group r for the changed range may be ob- 
tained from the actual r for that table: when 
this is multiplied by Noo (using the modified 
o’s which may have taken the value of 1 
the desired modified xy is obtained for en- 
tering into the compound. If one is reducing 
standard deviations, the Sxy will be smaller 
than the one which would be obtained by 
directly reducing the original Sxy in propor- 
tion. 


ALTERNATIVE PROCEDURES 


For the purpose of simple combinations, the 
tabulating of separate tables for all desired 
combinations, either by hand or machine, and 
the use of Dunlap’s combinative formulas, 
have already been commented on. It remains 
to note alternative procedures for holding 
means or standard deviations constant. Again, 
Dunlap’s formulas are available; to hold the 


Kelly, Truman Lee. Statistical Method (New York: M 
millan Co., 1923), pp. 221-30. 





sybgroup means constant, simply eliminate all 
‘the terms containing delta in his formula 
that is, the delta’s all become equal to 
zero. The delta’s may also be changed to 
-ome other value if desired. To equate or 
therwise modify standard deviations, multi- 
ply the appropriate terms in the numerator by 
the desired ratio, and the appropriate terms 
in the denominator by the square of the ra- 
tio. Other special results as described in the 
present paper can be secured from his for- 
mula. The relationship of his formula to the 
procedure in the present paper should be clear 
when it is recognized that multiplying out 
such a term aS Mox,, oy,,Tx,,y,, gives =xy for a 
subgroup; adding mé,,4,, shifts the origin to 
the compound group means. Similarly in the 
denominator, mo,,,* is =x* for a subgroup, and 
idding mé,,” shifts the origin to the compound 
roup mean. While the two procedures are 
interchangeable at various points, they start 
with different material, and make different de- 
mands. Dunlap’s formula assumes completed 
calculations for all of the tables; the method 
of the present paper is based on raw summa- 
and in some cases tabulations may be 
tirely avoided. 

May" presented a method of obtaining r 
when one starts with a compound table (or 
the essential parameters from it) and wishes 
to hold the means of the subgroups constant, 
without further tabulating. His method does 
not require linearity of the means, but re- 
juires their calculation. The method supple- 
ments that of the present paper, for it is 

pted to use with (the results of) a com- 
pound correlation table and does not require 
any of the constituent tables, whereas the 
nethod herein described employs summations 
from the individual tables entering into a 

mpound, and does not require a compound 
table. The choice between methods will there- 
fore depend on where one is beginning in his 
work. May’s method does not allow for modi- 
fying the individual standard deviations, and 
does not necessarily produce any tabulation. 


McCloy** presented a method of obtaining 
a compound of analytical correlations which 
equates both the means and the standard de- 


May, Mark A. “A Method for Correcting Coefficients of 
wrelation for Heterogeneity in the Data,’ Journal of Edu- 
ional Psychology, XX (September, 1929, pp. 417-23. 

McCloy, C. H. “A Method of Computing Partial Corre- 


tions, 


ation and hapeaten Equations with Variables Having Cur- 
ear Intercorrelations, ’ 


vember, 


Journal of Educational Research, 


XV] No 1927), pp. 285-95 
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viations of the subgroups. It does not assume 
linearity, and could be adapted to equate only 
the means or only the standard deviations, as 
desired. The procedure builds up a compound 
correlation table through special tabulating 
so that means and standard deviations are 
held constant by the method of tabulating. It 
does not yield separate correlations for the 
various subgroups unless these are especially 
made. It will however yield eta. 

Partial correlation, while usually thought of 
as analytical rather than combinative, is nev- 
ertheless an average—an average of specific or 
analytical correlations, with means equated; 
it is therefore comparable to the procedures 
described herein. It is definitely limited in 
two ways—through requiring linearity of 
means of subgroups when distributed on any 
pair of traits, and through requiring that the 
third variable be quantitative. It is true that 
arbitrary numbers may be used if the third 
variable is categorical, but these do not give 
satisfactory results unless they are so chosen 
as to lead to linear distributions of the means 
of subgroups or classes created by this third 
variable. Without this linearity the means of 
the classes are not held constant. The method 
does not lend itself readily to equating or 
otherwise modifying standard deviations. It 
is the only method of those discussed which 
is likely to prove practicable when mere than 
four variables are employed. On the other 
hand, when the third variable is quantitative 
and linearity is fulfilled, the method is 
adapted to expressing the average relationship 
between the third variable and the first, for 
various specific values of the second trait. 
The methods already described do not yield 
such comparisons. For traits one and two, 
however, additional variables can be held con- 
stant by one method as well as the other. 





All of these procedures will give the same 
results when the conditions which are as- 
sumed are fulfilled. 


SUMMARY 


The correlation coefficients, standard devia- 
tions, and means may be obtained for com- 
bined or separated populations by adding to- 
gether (or subtracting) the moment summa- 
tions, for the several constituent groups, which 
are ordinarily obtained for individual corre- 
lations: Sx’y’ or 3(x’ — y’)?, Sx”, Sy”, Sx’, 
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Yy’, and N. After such combination (or sub- 
traction) of the summations the results are 
ubstituted in the usual formula for calculating 
r from arbitrary working origins,—or in the 
appropriate formulas for the standard devia- 
tions or means. The procedure is not restricted 
to any particular correlation formula, nor to 
distributions which have been tabulated in 
two-way tables. It is somewhat more facile, 
when the necessary data are available, than 
ire combining formulas which utilize directly 
r,s, and M from each distribution. 

If the working origins and class intervals 
ire not the same for the tabulations of the 
several constituent populations, adjustments 
may be made for these differences. 


The method is well adapted to acco: 


ing certain special purposes, such as eli: 


ing variation in one or both means, and 


or both standard deviations, of the 


populations which are to be combined 


can be equated (held constant) by correct 


appropriate 


equal. These are forms of partial correlat 
which are not dependent upon a quanti 


moment summations for 
population before these are combined 
has the effect of giving an average corre] 
for the several groups as though one 

of the means or standard 


deviations 


third variable, nor upon assumptions of 
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